Bridging Theory and Practice Using Facebook: A Case Study

Abstract

In the past few years, Facebook has been increasingly studied for academic purposes due to its potential benefits to undergraduate education. Problems commonly found in university education include the lack of course engagement and the gap between theory and practice. This research contributes to the literature by investigating the impact of incorporating a non-mandatory Facebook group on learning outcomes. The analysis was done using a Taguchi method design, conducted with three two-level controlled factors (term, Facebook, and teacher). Results indicated that the students who participated in Facebook groups were more engaged with the course and applied theoretical knowledge to real-life problems better than students who learned under traditional instructional designs. Moreover, the use of Facebook groups led to better evaluation of teachers by the students. Furthermore, the student academic impact (knowledge and competence) was higher even though this complementary activity was not included in the course grade. We concluded that Facebook groups are excellent support tools that boost student engagement and their understanding of theoretical concepts and applying them in practice.

1. Introduction

The link between theory and practice is commonly referred to in the literature, however, one of the most frequent problems in university education is that students do not identify a connection between course topics and real-life situations or environments [1]. By incorporating real-life situations, the theoretical content is easier to understand and offers relevance to students [2]. Field trips are often highly relevant and help improve motivation, learning, and contextualization of knowledge [3]. However, the field visits are usually to regional industries and infrequently to national or international industries, therefore limited to specific industrial niches and often with a lack of edge technology, culminating in a generalized or irrelevant visit to the topics of the course. The foregoing does not favor the theory–practice relationship and does not help the development of transversal and disciplinary skills that organizations seek in students, because local or regional outlets do not help develop a global vision or offer sufficient knowledge to develop proposals of innovative and creative solutions [4].

Towards developing the necessary skills in university students and finding a theory–practice relationship, didactic techniques such as Problem-Based Learning (PBL) and Challenge-Based Learning (CBL) have been developed. These are based on sharing scenarios and solution examples from international organizations, using images, videos, and magazine articles that analyze real situations. Thus, after analyzing the cases, the students take ownership of the knowledge and develop critical thinking [5]. In addition, this methodology allows students to contrast their proposals with the solutions of their classmates, schoolmates, or even professional and industrial leaders [6]. However, in order to provoke adequate results, the shared materials must be contextualized and aligned to the desired topics [7]. In addition, care must be taken since there are uncontrollable factors that can alter the proposals and comments of the students, these factors can be personal (bad study habits, poor study strategies, and lack of personal motivation), institutional (educational model and course methodologies), and social (sociability, family environment, and socioeconomic context) [8,9,10,11].

In order to develop critical thinking in students and also to enrich their vocabulary, reading and writing activities must be developed [12]. Studies carried out by Channa et al. [13], demonstrated the lack of habits and attitudes of reading texts in university students. Due to the use of technology, such as phones, laptops, or tablets, students prefer ebooks and online accesses; in addition to the use of these devices as means of entertainment. As a result of the above, methodologies have been proposed with the use of electronic devices and social networks that help improve the academic performance of students during their leisure time [14,15].

The use of social networks has modernized education, specially with the worldwide COVID-19 pandemic that has underlined its value and turned it into a necessity [16]. Various social media for learning have been studied (Facebook, YouTube, Instagram, Twitter, WhatsApp, TikTok, Snapchat, among others), and although they show similarities and differences with the student performance, a deeper analysis should be carried out since there are several factors that can affect learning [17,18]. The social networks most used by students are Instagram, Facebook, and YouTube, being the first where students spend more time communicating, sending text messages, uploading images, and making video calls; Facebook is generally used to search for information, read news, and search entertainment, while YouTube is generally used to search for entertainment and self-improvement videos [19]. However, studies carried out by Rozgonjuk et al. [20], mention that Instagram can cause greater symptoms of disorder than Facebook. Instagram uses a more visual interface and ease of use; Facebook is usually a bit more complex, however, it offers features such as groups, pages, events, direct messages, broader statistics (followers, reactions, posts), etc., as well as allowing connectivity with other social media content [21]. Finally, Facebook offers tools and versatility, which would allow carrying out activities that offer support and correct behaviors in a more personal and private way [22], thus is a viable and preferable choice of a learning and teaching platform among younger generations [23].

To engage students in more active participation in their educational activities, improve and deliver deeper into knowledge, skills, and behaviors, we present a statistical analysis of the use of social networks together with CBL and PBL. Our research involved students enrolled in a Supply Chain and Logistics course of the bachelors in Industrial Engineering. This work uses the Taguchi method to evaluate the impact a new studying habit has when students receive academic content in Facebook videos or articles on their free time. Specifically, this study intends to measure students’ perception of the connection between theoretical course topics and their application to real-life problems. This proposal plans a non-invasive intervention with students because they decide whether to watch Facebook posts or not. Teachers and students are allowed to share content on a Facebook group specially created for this course. Posts must be related to the course topics but may include questions, comments, and reactions. The purpose is to enrich professor explanations by detailing concepts and analyzing different scenarios and points of view to generate meaningful learning. Results were analyzed to identify the degree of student engagement, their understanding of theory in practice, and the academic impact (knowledge and competence). Moreover, the impact on students’ perceptions of the teacher’s methodology and recommendations were analyzed.

2. Methodology

2.1. Statistical Analysis Model

A commonly used methodology, that employs a combination of mathematical and statistical techniques, is the Design of Experiments (DoE). Frequently used in experimental designs to develop efficient, balanced, and economical tests that allow to determine how controlled and uncontrolled factors are related to the process outputs [24]. Extensive applications of DoE can be found in the context of manufacturing settings but have also been translated into service industry [25]. In DoE, there is an area specifically dedicated to robust designs. Robust designs, proposed by Genichi Taguchi in the early 1950s, unlike classical DoE models recognize that not all factors that cause variability can be controlled. In general, the benefits of using Taguchi designs include: (a) reduced number of runs needed, (b) inclusion of noice factors, and (c) results consistency. Moreover, these models allow: (i) to determine the influence of factors and interaction under study, (ii) to identify the significant factors and their influences on the variability of results, and (iii) to determine the optimum condition for a product, a process, or a service along with an estimate of the contribution of individual factors and a prediction of the estimated response under the ideal conditions [24].

According to Carrillo-Cedillo Eugenia et al. [26], DoE can be very effective for solving problems in view of the new pedagogical challenges in engineering education. Recent research in education supported in experimental designs include: Lanas et al. [27], who were oriented to ask students about what they were doing, trying to engage them in theoretical reflection; Jiju et al. [28], who study the factors and their influences to improve students’ satisfaction scores during the delivery of teaching. By applying the Taguchi approach in this paper, we achieved two of the methodology’s objectives. First, the factors and their influences on the variability present in the course were identified. Second, the optimal conditions for teaching were established. This approach found that some of the control factors were the class timetable, materials used (slides, software, etc.), term of class, the social network used for learning, the teacher lecturing classes, and the psychology of the students and the instructor. All of these can affect educational quality but are often neglected in actual teaching standards [29].

To design this research, we used historical data obtained from continuous improvement tools managed by the academic department to determine factors that influence students’ outcome development. Results suggested that the primary sources of variation were associated with the teachers and their strategies and the term in which the class was lectured. The factors that were included in this experimental setting were the term of class (spring or fall), use of Facebook as a learning tool (included or not), and the professor who would lecture the class (1 or 2). Other factors that influence students’ outcomes are present in every teaching–learning process, but in this research, they were assumed to be part of the noise factors this methodology considers. With this in mind, the design experiment was an L-8 (2^3) without repetitions, carrying out a standard analysis because there was a single observation for each trial condition, and assuming that bigger is better.

2.2. Implementation Proposal

Participants were students enrolled in a Supply Chain and Logistics course in their senior year in the bachelor in Industrial Engineering program. All students who signed up for the courses during the spring and fall terms were asked to voluntarily participate. This study’s final count was 212, including 58 females (27%) and 154 males (73%). The participants’ ages varied from 21 to 29 years, with a mean age of 21.8, a standard deviation of 0.97, and a median of 22 years old. Before the start of this semester, 65% declared to have had an experience abroad, and 63% previously had a job, either an internship (93%) or a full-time job (7%).

All the sections selected for this study were scheduled at the same time, either on Mondays or Thursdays evening. Every course used Blackboard as a Learning Management System (LMS), on which the course teacher published announcements, syllabus, course materials, and resources. Quizzes and home assignments were submitted using Blackboard. Statistics-tracking powered by Blackboard was kept for all course materials and resources items. All sections followed similar course activities, including the same number and complexity of home assignments, case studies, two partial exams, one final exam, and a final project sponsored by a company. This course incorporated different instructional techniques, i.e., CBL and PBL.

One key performance metric at this University is the students’ recommendation about courses. A Students’ Opinion Survey (SOS), administered and applied by the University at the end of each semester and before final grades are released, is used to gather this information from every course. Survey responses are anonymous. Among the questions on this survey, two variables are of particular interest in this study: (i) the teacher recommendation (REC) for the course, and (ii) the course methodology evaluation (MET). Survey responses for these variables range from 0 to 10, with the possibility of not providing an answer to each question. An additional objective of this study was to maximize the students’ REC evaluation in this survey. Based on experience in this survey, a wide variety of factors may influence students’ REC. Some of the course-related factors include but are not limited to teacher, term, methodology, course difficulty, evaluation, and guidance, among others.

Teaching–learning is a complex process in which many factors may alter the output. According to Westwood [30], “very little can be done to modify most characteristics of learners (such as home background, poverty, health, disability, intelligence)”, which generates the need for robust sufficient teaching methods to make this process unresponsive to uncontrollable factors. This research proposes to use a Taguchi robust experiment design to make the process insensitive to environmental factors or other factors that are difficult to control [24]. Since no known factor combination consistently achieves the REC target value, some factors were taken as control factors. The rest were assumed to be still part of the system but considered as noise factors. The Taguchi method intends to optimize design parameters to minimize variation on the output parameter, in this case, the REC. Additionally, Taguchi’s orthogonal array method allows investigating the main effects of the selected factors even in the presence of uncontrolled (noise) variables.

For the present experimental work scope, a closed Facebook group was created and managed by the course teacher according to an L8 Taguchi design of experiments. The social network group was used in addition to the Blackboard LMS for those Facebook Approach (FbA) experimental sections. The purpose of the Facebook groups was to act as a complementary, non-mandatory course-resource repository for students. This is distinctive from other studies that implemented a Facebook group as part of the course assessment [31] or provided economic rewards [32]. Of the 105 students enrolled in FbA sections, 101 students (96.2%) freely joined the Facebook group at the beginning of the term. Students enrolled in FbA sections who did not join the Facebook group (3.8%) were left out of this study.

The section’s teacher and students were allowed to post on the Facebook group about anything related to logistics and supply chain. The teacher was in charge of reviewing and validating all posts’ quality and pertinence in the group. The Facebook group statistics were manually collected by two coders hired by the researchers in terms of quantity and category classification. The coders were trained and asked to code the posts separately. To test the reliability of coding, the inter-rater Kappa–Cohen coefficient of both coders was 0.93. A total of 1186 posts were published in the four FbA sections. Figure 1 shows examples of (a) student-to-student interaction, (b) student-to-teacher interaction, and (c) a teacher’s post. Of these 1186 posts, 7% were posted by teachers, averaging 22 posts per section. Posts were classified into the following categories: (a) (63%) “supply chain and logistics trends”, which included mainly news about the launch of new technologies in logistics or supply chain; (b) (21%) “Supply chain and logistics state-of-the-art”, which included reports of industry best practices and recent implementations of technology; (c) (14%) “Supply chain and logistics fundamentals”, which included explanations of key concepts; and (d) (2%) “other”, which included memes, surveys, and invitations to webinars and summits.

In the first session of the semester, a pre-test questionnaire was applied using surveys from the Blackboard course. The pre-test survey had no time limit or requirement to complete. Results from this survey evaluated the initial level of all sections regarding two primary constructs: (a) students’ engagement (SE) and (b) Students’ Understanding of Theory in Practice (SUTP). In the last session of the semester, before the final exam was applied, and final grades released, a post-test questionnaire was applied in all sections to evaluate the degree of change after the intervention. Students who participated in FbA sections were also asked about their experience during this experiment.

Items in both the pre- and post-test questionnaires were ratable on a 7-point Likert scale with answers varying from 1 to 7 (strongly disagree, disagree, slightly disagree, neutral, slightly agree, agree, strongly agree). Some of these items were considered answers in reverse code. To eliminate selection biases without administering differing versions of a Likert scale to some individuals in the group, we placed vertically oriented response options in the Blackboard surveys, as suggested by [33].

To assess the SE construct, we used the 22-items questionnaire proposed by [34]. This instrument is widely used [35,36] to measure agentic (e.g., “I tell the teacher what I like and what I don’t like”), behavioral (e.g., “The first time my teacher talks about a new topic, I listen carefully”), cognitive (e.g., “When I study, I try to connect what I am learning with my own experiences”), and emotional (e.g., “I enjoy learning new things in class”) engagement indexes. We chose this instrument because of its tested reliability and validity for population variety, reasonable length, and the measure of aggregate internal consistency. Internal consistency for (n = 208) pre-test responses was $\alpha$ = 0.94, while the post-test results for control sections (n = 107) was $\alpha$ = 0.95, and experimental sections (n = 101) was $\alpha$ = 0.94. These internal consistency levels are in alignment with the ones reported in [37].

On the other hand, to assess the SUTP construct, a 12-item questionnaire proposed by [38] was adapted. A wide variety of studies have used this instrument to evaluate reflective thinking [39,40]. In this study, items were grouped into three indexes, as shown in Appendix A, to gauge understanding, reflection, and critical reflection. Internal consistency for (n = 208) pre-test responses was $\alpha$ = 0.90, while the post-test results for control sections (n = 107) was $\alpha$ = 0.87, and experimental sections (n = 101) was $\alpha$ = 0.84. These internal consistency levels are higher than the ones reported in [40].

At the end of the semester, students enrolled in the FbA sections answered an evaluation survey shown in

Table 1. FbA experience questionnaire.

Q1. I understand the purpose of sharing on the Facebook group.

Q2. I felt comfortable using the Facebook group.

Q3. The Facebook group content reflected a practical application to this course.

Q4. The Facebook group was of some value to me.

Q5. The Facebook group was a useful learning aid.

Q6. The Facebook group stimulated my desire to learn.

Q7. The Facebook group might be used as a complementary activity in other courses.

Q8. Using the Facebook group was time demanding.

. Items in this questionnaire were adapted from [41] and from self-determination theory [42]. It also had a 7-point Likert scale with answers varying from 1 to 7 (strongly disagree, disagree, slightly disagree, neutral, slightly agree, agree, strongly agree).

At the end of the semester, student participation in the official Blackboard course space and the non-mandatory Facebook group was compared. Despite the positive effect that active student participation during class has on achieving higher grades [43], active student participation in the classroom was not considered part of this comparison. Studies on student participation in LMS indicate significant correlations between course grade and frequency-of-access variables [44]. Comparing the number of visits per item on the Blackboard LMS and the Facebook group will help determine any correlation with SE.

3. Results

In total, 208 students completed both the pre-and post-tests. Eight sections (runs 1 through 8) were considered with the variables indicated in

Table 2. Taguchi design.

Run	Term	FbA	Section’s Teacher	Enrolled Students
1	1	0	1	32
2	1	0	1	28
3	1	1	2	24
4	1	1	2	27
5	2	0	2	23
6	2	0	2	24
7	2	1	1	22
8	2	1	1	28

. Runs followed the L8 Taguchi method configuration, with the following three two-level factors: (a) term coded as 1 for spring and 2 for fall; (b) the FbA section indicator; if one, the section incorporated the Facebook Group and (c) the section’s teacher, 1 and 2. The number of students enrolled in the courses who completed the surveys is also provided.

It was possible to gather more information regarding these students from available university databases. These data included: (a) the number of Credits to complete before students’ Graduation (CtG), (b) the accumulated GPA (from 1 to 100, with 70 being the minimum passing grade), (c) student age at the beginning of the term, and (d) gender. Descriptive statistics are provided in

Table 3. Descriptive statistics for population demographic variables.

Runs	1	2	3	4	5	6	7	8
Section size	n = 32	n = 28	n = 24	n = 27	n = 23	n = 24	n = 22	n = 28
Gender = F	8 (25.0%)	14 (50.0%)	6 (25.0%)	3 (11.1%)	5 (21.7%)	5 (20.8%)	6 (27.3%)	10 (35.7%)
Gender = M	24 (75.0%)	14 (50.0%)	18 (75.0%)	24 (89.9%)	18 (78.3%)	19 (79.2%)	16 (72.7%)	18 (64.3%)
Age	22.0 (1.28)	21.6 (0.73)	22.2 (1.63)	21.6 (0.75)	21.7 (0.75)	21.9 (0.68)	21.9 (0.77)	21.8 (0.70)
CtG	12.8 (4.26)	11.8 (4.82)	10.9 (4.82)	11.4 (5.01)	13.3 (4.05)	11.4 (3.99)	11.2 (4.26)	11.9 (4.97)
GPA	84.6 (6.47)	84.9 (5.43)	86.6 (6.13)	87.1 (5.31)	87.1 (6.16)	84.3 (6.50)	84.9 (6.20)	86.0 (4.52)

Mean (SD) for Age, CtG, and GPA.

. To evaluate if there was an initial significant difference among groups in terms of age, CtG, and GPA, we made a series of hypothesis tests. Groups were tested for normality using the Anderson Darling test and homogeneity in variances using an F-test ratio.

Moreover, students signing up for these courses were independent, and they represented a sample of the industrial engineering campus population. Normality and homogeneity tests p-values are shown in

Table 4. Normality and t-test results for demographic variables across sections.

		Normality Test	p-Values for Age/CtG/GPA t-Test
			2	3	4	5	6	7	8
1	Age	0.369	0.298	0.636	0.153	0.636	0.757	0.886	0.697
	CtG	0.108	0.466	0.139	0.282	0.850	0.224	0.157	0.547
	GPA	0.335	0.800	0.270	0.110	0.140	0.840	0.780	0.260
2	Age	0.559		0.182	0.758	0.642	0.205	0.305	0.525
	CtG	0.388		0.562	0.780	0.258	0.580	0.784	0.882
	GPA	0.607		0.410	0.180	0.250	0.720	0.980	0.580
3	Age	0.741			0.097	0.405	0.850	0.778	0.423
	CtG	0.201			0.712	0.100	0.671	0.929	0.444
	GPA	0.680			0.600	0.780	0.210	0.400	0.770
4	Age	0.480				0.434	0.097	0.177	0.316
	CtG	0.424				0.165	0.985	0.888	0.698
	GPA	0.526				0.970	0.120	0.200	0.350
5	Age	0.714					0.473	0.591	0.918
	CtG	0.783					0.102	0.096	0.352
	GPA	0.652					0.130	0.230	0.490
6	Age	0.433						0.924	0.500
	CtG	0.425						0.886	0.632
	GPA	0.385						0.770	0.260
7	Age	0.705							0.627
	CtG	0.533							0.562
	GPA	0.641							0.450

. Note that the p-values are greater than the selected significance level 0.05, which implies that the data distribution is not significantly different from the normal distribution, i.e., the data are normally distributed. Additionally, there is no significant difference among the variances in the sections’ data, i.e., equality of the variances can be assumed. Moreover, p-values from all section test combinations indicate no initial significant mean difference among sections in terms of age, CtG, or accumulated GPA.

The pre- and post-test responses were on a 7-point Likert scale, i.e., ordinal data in which an ordering or ranking is possible, but no measure of distance is possible. Authors in [45] suggested using parametric statistical tests for analyzing ordinal data as interval data since they are more powerful than nonparametric tests. In the rest of the analyses, the authors assume that the psychological distance between “strongly agree” and “agree” is the same as between “agree” and “slightly agree” or between “slightly agree” and “neutral”. Additionally, the distance between “agree” and “strongly disagree” is five times greater than between “agree” and “strongly agree”. Data analyses for the pre-test questionnaire were carried out for a total of n = 208 responses. The SE and SUTP construct values for each group were calculated as the average of the indexes’ average. In Figure 2, the SE and SUTP constructs for pre-test and post-test are compared. Despite control sections 2 and 6 experiencing an increase in one of the two constructs, no consistent improvements were achieved on control groups. However, the SE and SUTP constructs in all FbA sections (runs 3, 4, 7, and 8) increased at least 4.7% and 4.1%, respectively.

The mean (and standard deviation) reported as the average of items in each index, and the Cronbach’s alpha reliability coefficients for SE and SUTP were calculated.

Table 5. Internal consistencies and descriptive statistics for pre- and post-test indexes’ values across sections.

Construct	Index	Test	$\mathit{\alpha }$			Runs
				FbA		1	2	3	4	5	6	7	8
						FbA
				0	1	0	0	1	1	0	0	1	1
SE	Agentic	Pre	0.84	4.46 (1.22)	4.37 (1.25)	4.66 (1.1)	4.19 (1.27)	4.68 (1.46)	4.36 (1.11)	4.22 (1.3)	4.74 (1.22)	3.96 (1.31)	4.44 (1.14)
	(5 items)	Post	0.87	4.09 (1.43)	4.77 (1.05)	3.96 (1.51)	4.26 (1.43)	5.08 (0.92)	4.41 (1.15)	4.18 (1.49)	3.95 (1.32)	4.55 (1.05)	5.03 (0.96)
	Behavioral	Pre	0.81	4.26 (1.16)	4.41 (1.21)	4.04 (1.08)	4.16 (1.06)	4.36 (1.19)	4.5 (1.17)	4.57 (1.12)	4.4 (1.41)	4.11 (1.18)	4.59 (1.34)
	(5 items)	Post	0.81	4.04 (1.37)	4.93 (0.84)	3.91 (1.29)	4.22 (1.5)	5.14 (0.92)	4.79 (0.7)	4 (1.39)	4.04 (1.39)	4.75 (0.73)	5.01 (0.95)
	Emotional	Pre	0.81	4.37 (1.18)	4.22 (1.33)	4.53 (0.96)	4.2 (1.27)	4.47 (1.34)	4.24 (1.28)	4.28 (1.17)	4.44 (1.38)	3.89 (1.35)	4.26 (1.38)
	(4 items)	Post	0.79	4.05 (1.43)	4.76 (0.9)	3.8 (1.49)	4.29 (1.4)	5.01 (0.93)	4.44 (0.83)	3.96 (1.49)	4.18 (1.34)	4.75 (0.95)	4.87 (0.86)
	Cognitive	Pre	0.79	4.42 (1.01)	4.40 (0.99)	4.55 (0.77)	4.08 (1.05)	4.65 (0.95)	4.52 (0.97)	4.39 (0.97)	4.67 (1.23)	3.95 (1.05)	4.43 (0.95)
	(8 items)	Post	0.86	4.04 (1.23)	5.0 (0.87)	3.94 (1.25)	4.17 (1.4)	5.32 (0.81)	4.79 (0.98)	4.11 (1.25)	3.93 (1.05)	4.71 (0.84)	5.16 (0.75)
SUTP	Understanding	Pre	0.86	4.29 (1.48)	4.5 (1.4)	4.05 (1.45)	4.69 (1.31)	4.55 (1.16)	4.48 (1.45)	3.85 (1.34)	4.57 (1.72)	4.47 (1.24)	4.51 (1.72)
	(4 items)	Post	0.9	4.14 (1.44)	4.87 (0.98)	4.08 (1.31)	4.11 (1.38)	4.89 (0.84)	4.64 (1.24)	3.66 (1.72)	4.7 (1.31)	5.1 (0.82)	4.9 (0.92)
	Reflection	Pre	0.64	4.32 (1.23)	4.37 (1.31)	4.22 (1.26)	4.7 (0.98)	4.69 (1.2)	4.3 (1.32)	3.91 (1.1)	4.4 (1.47)	4.25 (1.23)	4.27 (1.49)
	(4 items)	Post	0.88	4.09 (1.47)	5.05 (1.16)	3.81 (1.45)	4.16 (1.31)	4.79 (0.86)	4.82 (1.44)	3.75 (1.59)	4.69 (1.46)	5.32 (0.91)	5.27 (1.24)
	Critical Reflection	Pre	0.72	4.35 (1.24)	4.46 (1.26)	4.23 (1.31)	4.55 (0.89)	4.83 (1.32)	4.28 (0.99)	4.01 (1.35)	4.6 (1.37)	4.38 (1.2)	4.38 (1.49)
	(4 items)	Post	0.84	3.98 (1.28)	5.08 (0.99)	3.8 (1.21)	3.92 (1.14)	4.97 (0.97)	4.87 (1.11)	3.67 (1.5)	4.58 (1.2)	5.33 (0.93)	5.18 (0.94)

Mean (Standard Deviation).

shows the comparison of the pre- and post-test per index per section. Pre-test results indicated no statistical mean difference for any of the evaluated indexes among sections, which was expected because sections were randomly selected. However, notice in the post-test results that the index’ mean is at least 8% greater than the pre-test index’ mean for every index in all FbA sections, meaning the use of Facebook positively affects both the SE and SUTP. Moreover, all FbA sections reduced (at least 11%) their standard deviations when comparing pre- and post-test values for all FbA sections. Strong correlations were found within SE indexes (0.69 < r < 0.86), as well as within SUTP indexes (0.82 < r < 0.89). However, no correlation was found when comparing SE and SUTP indexes, which means an increase in SE will not necessarily impact SUTP and vice versa.

To gauge the degree of change in the two constructs measured in this research, we calculated the difference between the post-test and the pre-test results for each participant’s index. A positive difference means an increase in the index. A commonly used statistical test to contrast the impact of a treatment is the paired t-test. A series of paired t-test and paired F ratio was performed for each index and each section. The first one was to test for an increase in the index’s mean on the post-test results and the latter to evaluate a standard deviation reduction. The results from paired tests are shown in

Table 6. Paired difference test for index’ mean and standard deviation across sections.

Index	Runs
	1	2	3	4	5	6	7	8
Agentic	0.018 *	0.576	0.878	0.572	0.468	0.021 *	0.940	0.972
	(0.041 *)	(0.266)	(0.984)	(0.429)	(0.264)	(0.362)	(0.842)	(0.802)
Behavioral	0.316	0.576	0.986	0.857	0.076	0.212	0.963	0.875
	(0.164)	(0.039 *)	(0.884)	(0.994)	(0.164)	(0.525)	(0.983)	(0.960)
Emotional	0.018 *	0.596	0.941	0.749	0.253	0.252	0.982	0.951
	(0.008 **)	(0.303)	(0.955)	(0.984)	(0.13)	(0.546)	(0.944)	(0.992)
Cognitive	0.021 *	0.618	0.998	0.826	0.233	0.029 *	0.991	0.997
	(0.005 **)	(0.067)	(0.775)	(0.474)	(0.12)	(0.770)	(0.846)	(0.880)
Understanding	0.532	0.045 *	0.933	0.674	0.334	0.610	0.985	0.841
	(0.718)	(0.407)	(0.933)	(0.788)	(0.127)	(0.898)	(0.967)	(0.999)
Reflection	0.133	0.045 *	0.677	0.915	0.339	0.741	1.00	0.995
	(0.215)	(0.069)	(0.940)	(0.329)	(0.048 *)	(0.518)	(0.916)	(0.827)
Critical Reflection	0.09 *	0.015 *	0.714	0.971	0.165	0.478	1.00	0.986
	(0.666)	(0.098)	(0.926)	(0.277)	(0.309)	(0.742)	(0.875)	(0.990)

p-value for mean (p-value for SD); * p < 0.05, ** p < 0.01.

. Despite the course’s instructional design having been carefully planned and designed to include different teaching techniques (PBL and CBL), the students enrolled in the control sections were not considered to have a significant mean increase on some indexes. This means that neither the course design nor the teacher is consistently developing the SE or SUTP. However, the responses from students who experienced the FbA treatment indicated a significant statistical increase in the mean values of all indexes. Moreover, a statistically significant decrease in the standard deviation index mean was achieved in all sections that implemented FbA. Results indicated that the FbA sections achieved greater mean values of SE and SUTP with less dispersion when compared to initial values.

Once the semester ended, Blackboard LMS statistics from all sections and Facebook group statistics from FbA groups were retrieved. Blackboard items that impacted the final grade were discarded to make this comparison since the Facebook group was optional in FbA sections.

This analysis was divided into two parts: the first part comparing the Blackboard statistics across all sections, and the second part, comparing Facebook with Blackboard for the FbA sections. For the first part, for all sections, the percentage of students that accessed each item on Blackboard was compared, using the Blackboard statistics tracking feature. The number of items per section ranged from 26 to 33, with a mean of 30.2 and a standard deviation of 2.1 items. The average percentage of students that accessed items was calculated for each section. The item with the lowest access (21%) on all sections was the same: course syllabus, a document that explains the connection between learning outcomes and course content, bibliography, grading, responsibilities, rules, and expectations). The teachers suggest that the reason why syllabus access might be the least accessed is that they explain this document in the first session of the semester. The authors decided to exclude the course syllabus items from the rest of the analyses to avoid bias. The average access to sections ranged from 45% to 97%, with a mean of 63.4% and a standard deviation of 7.3%. A series of t-tests were made to explore any potential mean difference in the percentage-of-item access. Results indicated that the mean percentage of students that accessed items in all sections was the same since no statistical difference among sections was found (all p-values were greater than the significance level 0.05).

For the second part, the FbA sections were considered. The same two coders collected the data. The inter-rater Kappa–Cohen coefficient of both coders for the “seen by” variable was 0.99. The percentage of students that accessed each Facebook group post, identified by the “seen by” Facebook feature, was calculated for each FbA section. The number of posts per section ranged from 260 to 341, with a mean of 297 and a standard deviation of 33 posts. The average percentage of students that accessed (“seen by”) posts was calculated for each section. Average “seen by” percent on sections ranged from 65% to 100%, with a mean of 81.9% and a standard deviation of 5.1%. An increase in mean item access of 18.5% was achieved when comparing the Facebook group with the Blackboard space, meaning that academic content is reaching more students via a Facebook group. A series of t-tests were made on the FbA sections to explore a potential mean difference in post access percentage. A higher percentage of students accessed the optional Facebook group than the official Blackboard space (p-values 0.65, 0.73, 0.45, and 0.68 for sections 3, 4, 7, and 8, respectively).

The impact of this experiment was also tested on the final course grade. The average students’ GPA for control sections (n = 107, mean = 85.7, and sd = 6.16) and for FbA sections (n = 101, mean = 86.2, and sd = 5.5) were compared to the final grades mean and (standard deviation) for the control sections [83.4 (9.32)] and FbA sections [85.8 (8.8)]. Despite that, control group students, on average, achieved lower grades than their overall GPA in this course, the mean difference for the control groups was −2.06%, while for the FbA sections, it was −0.5%. A paired difference t-test comparing GPA and final grade was made to test the control and FbA sections. The control sections experienced a significant change in the mean (p-value 0.011), while for the FbA sections, no significant mean difference was found (p-value 0.483). The results indicated that the use of FbA positively impacted achieving higher grades. Researchers found a positive correlation (r = 0.31) between the (final course grade minus accumulated GPA) and the amount of Facebook group posts. Moreover, a solid positive correlation (r = 0.73) was found in students with GPA < 80, meaning that the students with historical lower performance found the required course engagement and understanding of topics to achieve higher final grades. This result is consistent with [44], who also found a significant correlation between course grade and access to course content.

Additionally, researchers noticed that students tended to post more actively as certain milestones approached (the first and second partial exams and the final exam). The number of accumulated posts during each of the three time periods divided by the milestones were studied. Pearson’s correlation test was performed to evaluate the inverse relationship between the accumulated posts and the number of days to a milestone. For each FbA section, the r coefficients for the three time periods between two evaluation milestones are shown in

Table 7. Pearson correlation coefficient between accumulated posts and days-to-milestone per section.

Section	Time between Milestones
	First Day of the Semester to First Partial Exam Day	A Day after First Partial Exam to Second Partial Exam Day	A Day after Second Partial Exam to Final Exam Day
3	−0.75	−0.85	−0.69
4	−0.77	−0.94	−0.81
7	−0.78	−0.74	−0.86
8	−0.84	−0.95	−0.91

. Results indicated a strong negative relationship for every section. Thus, the data indicated that the sharing behavior of students was more active when the exam date was closer in time, which suggests that they were using non-traditional approaches to study [46].

Students’ Opinion Survey (SOS) results for the REC and MET variables for every section were used as response variables in the Taguchi method design. The SOS data analyses indicated an increase of 12% in the MET mean value for those sections where FbA was incorporated. Additionally, statistical tests found no significant effect of either term or teacher, as shown in Figure 3.

One of the most relevant SOS questions for this university is whether the students would recommend a friend taking this course with this teacher. The Taguchi analysis results indicated that incorporating the FbA will positively shift this recommendation without considering the teacher or the term, as shown in Figure 3. Note an increase in the REC mean value of 18% when FbA is incorporated. These results indicated that despite other noise factors not being controlled, the implementation of FbA consistently helped teachers improve their MET and REC evaluation on the SOS.

The responses from the FbA evaluation survey are shown in Figure 4. Results higher than four indicated a positive student evaluation, while answers smaller than four suggested a negative student evaluation. In general, the Facebook group was well perceived by students. At most, 10% of the students fell short of their expectations on questions (Q1 through Q6) related to FbA purpose, use, content, and learning support. Special attention needs to be directed to Q5 with the highest positive evaluation (79%) and the highest percent of strong agreement expressed by students on the statement, “Facebook group was a useful learning aid”. On the other hand, despite 43% of the students not agreeing that this activity is time demanding (Q8), and 71% claim it to be of some value (Q4), 39% would not recommend implementing this activity in other courses (Q7).

4. Discussion

The purpose of this research was to investigate what effect incorporating an additional, non-mandatory Facebook group in a university course has on student engagement and understanding of theory applied in practice. Despite the literature controversy about whether social networks do or do not improve the teaching–learning process, some cases have proven to be a great help [47], and some have demonstrated a negative impact [48]. The authors predicted that an extra course resource, such as a Facebook group, could leverage its features to achieve greater levels of SE and SUTP. The findings obtained in this study, from widely used instruments that measure engagement [49], indicated that the students exposed to the use of FbA had more course engagement in all the indexes (agentic, behavioral, emotional, and cognitive) than those students who took courses without a Facebook group [22]. This is also consistent with Northey et al. [50], who state that “incorporating an asynchronous component to enable more porous boundaries and creating a learning ecosystem resonates with students, as shown by the significant increase in engagement”.

On the other hand, the authors implemented this experiment to increase SUTP. Facebook group posts intended to provide students a mechanism to link coursework to practice through discussion and collaboration. Bosman et al. [51] shows how integrating online discussions can have positive learning implications and provide students the opportunity to connect real-world with theoretical underpinnings, therefore reducing the prevalent unacceptable gap between theory and practice [52]. The findings obtained from the SUTP construct indicated that FbA sections consistently improved the pre–post evaluations. Results obtained from a correlation test indicated no relation significant correlation between SE and SUTP; however, internal consistency was shown in the evaluation instruments.

It was expected that students in both the control and FbA sections who have had an experience abroad or a job would have a higher pre-test SUTP since they had been exposed to more and different learning conditions. Hypothesis tests were performed to evaluate potential mean differences in students with these experiences versus students without them. For the job experience, the mean SUTP pre-test results of students who had a job experience in sections 2, 4, 6, 7, and 8 (p-values 0.041, 0.012, 0.037, 0.024, and 0.034, respectively) were higher than the students without job experience. However, in the post-test results, there was a statistical difference in section 6 (p-value 0.015). This indicates that despite a job providing students a clear panorama of industry problems, students in FbA sections without working experience had no SUTP difference from those with a job opportunity. Similar tests were performed comparing students with and without an exchange abroad in their undergraduate studies. A significant mean difference for the pre-test results was found in sections 2 and 7 (p-values 0.035 and 0.028, respectively), with a higher mean for the students who went abroad. The post-test results indicated no difference in the mean SUTP between students with experience abroad in all sections.

Sections using FbA had experiences that, on average, helped the students to increase their final grades in this course. However, in the absence of those experiences, i.e., the control groups, no significant increase in final grades was present. Alshuaibi et al. [36] suggest implementing social media as a learning environment to promote students’ cognitive engagement and academic performance, consistent with this study’s findings. Moreover, despite evidence showing no difference in Blackboard activity for both FbA and control sections, the teachers in charge of the sections observed that students were more engaged and active during their FbA sessions. These teachers’ statements are consistent with the data regarding students’ access to Facebook posts, which is significantly higher than the access to Blackboard.

Arteaga Sánchez et al. [53] findings indicate the positive impact that Facebook has on student academic performance. In this study, we also found a positive correlation (r = 0.31) between the number of Facebook posts and the difference between final course grades and the accumulated GPA. This suggests that students who became more engaged in FbA groups, also performed greater than they did in the past courses. Interestingly, we found that the correlation is even higher (r = 0.73) for those students with lower-than-average GPAs. Implementing Web 2.0 technologies, such as in FbA groups, helps students improve in general but particularly those low-achievement students [54]. This is achieved by improving learner motivation and encouraging self-empowered learners. Teachers from the FbA strongly suggest other educational professionals willing to replicate this experiment take leadership and become involved in Facebook group discussions.

Finally, results indicated that students in the FbA sections gave higher scores for the MET (methodology used) and REC (course recommendation) variables than those in the control sections. Other studies implementing a Facebook-related activity have also found that greater course engagement is associated with higher course satisfaction [37]. Implementing a Taguchi method design showed that other factors, such as teacher or term, had no significant effect compared to FbA factors in the MET and REC response variables. The SOS results were consistent with the FbA evaluation survey, as students claimed that the Facebook group was a useful, non-time demanding, valuable, complementary activity that stimulated their desire for learning. However, students do not recommend the use of this activity in all courses. This last statement was also tested for other courses and term lengths.

The authors had tested this same course setting in more technical/mathematical courses where SUTP is not straightforward, such as Operations Research (OR). The FbA results were not replicable. A possible explanation is that Facebook content could not reflect direct and realistic application (Q3) of the OR course. Furthermore, the FbA design was also applied to an intensive summer course, the same one described in this study. The authors found that the summer students were charged with too many activities in a short time, so they considered the Facebook group to be a time demanding activity (Q8).

5. Conclusions

The present study is part of the authors’ intent to continue contributing to literature that examines the impact of implementing Facebook groups in undergraduate courses, demonstrating that using this additional resource correlates to increasing student engagement and understanding of applying theory in practice. Results indicated that students used their free time to watch videos, read articles, and discuss topics with their classmates and the teacher in a Facebook group. This supplementary activity helped the students connect theoretical course content and practical application to solve real-life problems. Data analyses indicated a close relationship between students’ perception of the teacher, engagement, and learning experience. A Taguchi method design helped consider the optimal controllable factors that mostly influence the students’ course recommendation. Findings in this research suggest that Facebook groups can also lead to academic impact (knowledge and competence), especially among those students with the lowest GPAs. It would be interesting to continue research to find the causes of these findings.