Taking Advantage of Students’ Passion for Apps in Sustainability and CSR Teaching

Abstract

The main aim of this study was to evaluate the effectiveness of using mobile apps combined with traditional methods for teaching sustainability and corporate social responsibility (CSR) to undergraduate students. Although previous papers stated the importance of apps, they showed no conclusive results, and there are few studies analyzing their importance in management training, and particularly in sustainability management. To achieve our main goal, drawing upon the action learning system, we designed and implemented several training sessions combining traditional methods and apps. The effectiveness of teaching methods was assessed with two surveys (pre- and post-training) with questions about sustainable consumption, lifestyle, and CSR management. The results demonstrate that students found this type of learning beneficial, showing an increase in their commitment to social and environmental issues. These findings underline the potential for apps in teaching sustainability at undergraduate levels in combination with more traditional methods.

1. Introduction

On the first day of their management courses of the 2017–2018 academic year, students were asked how many of them had a smartphone. Their answer was unanimous: everyone. Then, they were asked about how many of them used mobile applications (apps) in their daily lives. The reply was the same (everyone). Smartphones have become an essential part of our everyday life [1] and are regarded as flexible tools that facilitate access to information in different formats and in any environment [2]. Each new version of these devices brings innovative features that make them more convenient and affordable, and new apps continually become available that make our lives easier. Currently, 96% of smartphone users can access the internet without a Wi-Fi connection [3]. As of the third quarter of 2018, two million mobile apps were available to download from Apple’s App Store. In the same period, there were 2.1 million Android apps on Google Play [4]. These advances prompted educators and researchers to utilize these devices to promote teaching and learning [5]. There is great potential in using mobile devices to transform how we learn by changing the traditional classroom to one that is more interactive and engaging [6].

Recent empirical evidence indicates that mobile devices can be used to support students´ learning in higher education settings [7,8,9]. Despite this, Pimmer, Mateescu, and Gröhbiel [10] (p. 492) reported that “after more than 20 years of mobile learning research, there is still relatively little systematic knowledge available, especially regarding the use of mobile technology in higher education settings”. Moreover, the findings from previous literature are contradictory, since papers reported mixed results on the effect of mobile environments on learning outcomes [5]. On the one hand, some studies reported improvements in students´ learning achievements (see, for example, References [11,12]). On the other hand, others studies found that there was not sufficient evidence on whether mobile games improved learning outcomes (see, for example, References [13,14]). Additionally, regarding the subject matter domains, Crompton and Burke [7] conducted a systematic review of mobile learning research during 2010–2016 in higher education settings, and their findings revealed that the most often researched subject matter domain was second language instruction (21% of the studies) followed by education topics (14% of the studies). They also reported that only 8% of previous research was focused on business matters.

The scant research that analyzed the effectiveness of these learning methods in business matters contrasts with the existing literature on management education, and specifically on sustainability management education, highlighting the necessity of incorporating new action learning methods into business courses. In particular, in higher education, the challenge is to ensure that business students leave university with a deep understanding of their role and responsibility toward their future employers, community, and society in general [15]. Several schools, aware of this necessity, are teaching these topics using experiential learning and immersion techniques [16]. Figueiró and Raufflet [17] asserted that traditional methods of management education, focusing on a single and isolated discipline and often based on unidirectional education processes, fail to provide the training for graduates to work toward developing solutions around sustainability issues in a new and complex world. Barth, Godemann, Rieckman, and Stoltenberg [18] highlighted that this requires an essentially new learning culture, incorporating an open-minded and participative process. Action learning methods represent a very promising methodology for teaching sustainability as it involves student participation in problematization, research, problem solving, and critical reflection, using different interactive tools. These learning methods generate cognitive engagement, which can increase students’ motivation and develop their critical thinking skills, thoughts, and values in sustainability [19]. The lack of consensus in previous literature may be because the most appropriate methods for training in management go through a combination of both traditional and new methodologies. In line with this, Reynolds and Vince [20] proposed that management education could benefit from bringing together two distinct methods within management learning (traditional perspectives to management theory and action-based learning). Other authors insist on bringing up that mixed teaching methods, i.e., those that combine traditional training (textbooks, handbooks, scientific journals, etc.) with digital tools (apps, videos, other technology aids, etc.), make it possible to empower a more intensive learning in sustainability issues [3,21]. In line with this, although there are numerous papers suggesting the importance of apps in learning across a range of disciplines (see Reference [22] for a review of these), there are few studies analyzing the importance of apps related to sustainability in management training [23]. Specifically, as far as we are aware, there are no empirical studies about the impact of apps in changing student behavior and perception in the field of sustainability; in this way, we also contribute empirically to the previous literature.

With these four identified gaps in academic understanding (lack of systematic knowledge available in mobile learning research, mixed results in previous literature, scarcity of studies focused on business and sustainability topics, and the absence of consensus about appropriate teaching methods), this paper discusses the potential of using apps as an active teaching strategic method to train students about sustainability and corporate social responsibility (CSR) topics. In particular, the research question was as follows: how and to what extent can the use of apps combined with traditional methods contribute to the development of knowledge, skills, and the values of sustainability and CSR in undergraduate students?

In order to answer the above research question and to contribute to the previous literature, in this paper, we combine traditional methods of teaching CSR (traditional lessons, news, reports, etc.) with active methods such as technology aids based on specific CSR mobile applications. This article focuses on two apps—Abouit and Label for your Planet—that were developed for tablets and smartphones. Most teachers view access to the internet and educational apps as the most beneficial features/functions of mobile phones for classroom use [24]. Apps offer user-friendly information that is deployed for training in many disciplines [25,26].

The rest of paper is organized as follows: Section 2 provides the theoretical background of using apps in management courses to reinforce students’ sustainable behavior. Section 3 presents the research design, including the sample, the variables, a description of the initiative carried out, and the research methods used to evaluate its effectiveness. The results are presented in Section 4. Finally, the discussion and conclusions, as well as promising avenues for future research, are summarized in Section 5.

2. Theoretical Background

In management education, increasing attention was paid to sustainability over the past 10 years [17], and sustainable environmental education is a vital issue in our time [27]. In higher education, the challenge is to ensure business students leave university with a deep understanding of their role and responsibility toward their future employers, community, and society in general [15]. Additionally, they should be aware of the implications of being a sustainable professional and the important part they play in shaping a business world that is perceived as contributory and responsible [28].

A question regarding teaching sustainability is the existing debate in the previous literature about the final purpose of the training. Aragon-Correa, Marcus, Rivera, and Kenworthy [29] identified the tension between teaching sustainability to improve company financial performance and to change the world for the better. The first is related to an instrumental stakeholder-based view of the firm. This perspective holds that an organization can be seen as a set of interdependent relationships between its stakeholders [30], who are jointly committed to its success. In this case, students need to learn how to assess, create, measure, and then effectively communicate the financial benefits of sustainable practices.

Regarding changing the world for the better, previous research pointed out that teaching resources in sustainability should be focused on the moral imperative of making the world a superior place [29]. Thus, the challenge facing business schools is to develop responsible leaders who are prepared to deal with complex and value-laden issues in the economy and society [31], not merely to turn out students as managers of successful sustainable firms. Teaching sustainability should compel analyses of global environmental and social responsibility issues. These problems include working conditions, discrimination, climate change, water conservation, income, jobs, and so on [29]. Exploring this tension between making companies better off and changing the world for the better is critical to moving education forward in the sustainability domain [32]. Aragon-Correa, Marcus, Rivera, and Kenworthy [29] carried out a descriptive study using a sample of management instructors specializing in sustainability, and found that the moral imperative of changing the world for the better eclipsed the pragmatic imperative of improving company performance. Their final suggestion was that instructors create engaging, innovative, and effective teaching materials that provide a business case for sustainability, while illustrating the moral and ethical imperatives incumbent on sustainable business practices.

Furthermore, regarding teaching sustainability, Stubbs and Cocklin [33] highlighted the need to develop students’ ability to think in new ways and from different worldviews. As Kurland, Michaud, Best, Wohldmann, Cox, Pontikis, and Vasishth [34] (p. 459) stated, “if the world demanded that decisions be made in fundamentally different ways, then [it follows that] we should educate students in quite different ways as well”. Students should, thus, be stimulated to be active learners instead of mere knowledge consumers [35]. A dynamic mutual learning process is required in order to learn from experiencing the process instead of purely memorizing its characteristics [36]. Sustainability teaching explores the connection of humans with nature and requires the combination of analytical, physical, and spiritual concepts and practices into a holistic learning experience [32]. Teaching strategies adopted by lecturers influence students’ learning and, as many authors argue, a more interactive learning style is needed [37]. Active teaching, such as case methods, simulations, or games, are more effective for students to develop managerial competences [31].

Therefore, there is a consensus in the literature on the clear need to revise classic teaching methods for sustainability; a thorough discussion and critical assessment of the advantages, limitations, and shortcomings of each alternative technique is also required. “Because sustainability is, by its nature, a concept and topic that calls for action, the active learning approaches preferred by students may be more valuable [in relation to classical teaching with a passive learning process] in this emerging focus of business inquiry” [38] (p. 204). Figueiró and Raufflet [17] analyzed new learning techniques such as action/experiential learning, service learning, and problem-based learning, among others, which are used in “transformative learning”, an educational strategy that incorporates interactive methods of teaching and learning, transforming the learning process from passive to active [39]. Thus, new methods for teaching in sustainability seem to be required, and the action learning approach is emerging as a very promising method to solve this novelty. It involves the students’ participation in problematization, research, problem solving, and critical reflection, using tools such as teamwork, case studies, projects, discussions, apps, and games.

Consequently, there can be no doubt that today, more than ever, instructive and effective teaching materials that both inspire management students and provide them with specific tools and techniques are needed to tackle moral challenges for making the world a better place [40]. Similarly, Juárez-Nájera, Dieleman, and Turpin-Marion [35] argued that students should be stimulated to be active learners instead of mere knowledge consumers. The combination of traditional methods with mobile apps can help educators stimulate students to be more active and to teach them not only to be responsible managers, but also sustainable citizens.

Apps are an interactive method of teaching and learning that transform the learning process from passive to active [39] and involve activities in which students learn by doing [17]. Apps present a significant educational opportunity when they are employed appropriately [41], and can be extremely useful tools for working with a range of age groups [5]: children [42], adolescents [43], and people with psychological, psychomotor, and physical difficulties [44]. It was shown that active teaching strategies stimulate students’ participation and interaction in the classroom [45], and allow them to be integrated with daily tasks and work experience [20]. All these features should make the use of apps an ideal way to improve contemporary, traditional learning methods [46,47]. When it comes to management teaching, technological progress, and particularly the explosion of apps, makes it easier for students to connect more immediately with problems in society and in companies, which is especially helpful for training in sustainability and CSR. Information about sustainable companies and products, easily obtainable from an app, enables students to assimilate knowledge about business strategies, as well as practice their skills in the sustainability field.

By using “sustainable apps", students tacitly acquire skills that they can then apply to their businesses. The manner in which they imbibe this learning is also important, given that visual capacitation through images makes it easier to absorb the message. What is more, collating information about fast-moving consumer products brings students into the real world; that is, it gives them an awareness of the problems of the world around them, consistent with the premise of changing the world for the better. Moreover, these applications allow students to discover how they could manage problems deriving from the incorrect positioning of a company’s product, about decisions they would have to make and why they should bear these apps in mind, together with the information that appears on them, in order to get a better position in the market.

Affordances of mobile technologies are used in a variety of learning activities, including but not limited to as a multimedia access tool for learning by exploring and reflecting, a multimedia collection tool for learning by visualizing and reflecting, a communication tool for learning by conversing, a connectivity tool for learning by sharing, a representation tool for learning by visualizing, and a knowledge construction tool for learning by constructing [3]. Previous results indicate that, although apps have the potential to facilitate learning during traditional teaching such as case studies [48], further knowledge about the topic is required in order to use technology tools most effectively. The meta-analysis of Haβler, Major, and Hennessy [49] supports the view that students’ learning outcomes will improve when gaining knowledge via traditional education is assisted by technology. In line with this, Fernández-López, Rodríguez-Fórtiz, Rodríguez-Almendros, and Martínez-Segura [50] also found positive learning outcomes when using their learning apps on mobile devices combined with traditional methodologies in a sample of children with special education needs. Also, as Carlson [51] suggests, millennials consume and learn from a wide variety of media, often simultaneously. Similarly, other authors believe that there is substantial evidence that incorporating technology of any kind in the traditional classroom as an instructional tool enhances student learning and educational outcomes [52]. This mixed-methods model is advantageous to action teachers because it “can result in well-validated and substantiated findings” [53] (pp. 213–214). With this approach to sustainability and CSR teaching, it seems that a combination of this type of method using new technologies and more traditional methods is needed to improve students’ education and encourage their scope for learning. Mixed-application tactics, such as using “high-tech” tools (internet, software applications, monitors, and sensors) to support “low-tech” (traditional) tools, are not only possible, but also often desirable [21]. However, the use of “high-tech” tools alone would be insufficient to foster learning without the adoption of appropriate pedagogies [3]. On the basis of these ideas, we propose the following hypothesis:

Hypothesis 1 (H1).

The use of mixed teaching methods (traditional and apps) contributes to improving students’ learning in CSR and sustainability topics.

3. Research Design

3.1. Sample and Data Collection

This research used a sample made up of undergraduate students at a Spanish university involved in different degrees in the Faculty of Economics and Business Studies, who are enrolled in Business Strategy courses. We focused our research on these students because some studies state that, traditionally, business students are taught that every facet of what they do is essentially economic [54,55] and they put more emphasis on efficiency and, particularly, on maximizing personal material outputs [56,57,58]. This type of student tends to show more non-cooperative behavior [59,60], is significantly more corrupt than others [61], and cheats more that their non-business peers in order to achieve his/her goals [62]. Therefore, it would be particularly interesting to try to make these students aware of the importance of being social and environmentally conscious [63,64]. Furthermore, Business Strategy courses, and specifically topics related to CSR and sustainability, are mainly taught in the aforementioned faculty, which may make these students suitable for developing initiatives such as the one described in the present research.

The sample (students who returned both completed surveys employed in our initiative as commented below) was made up of 127 people, of which 82 were pursuing a Business Administration degree (BA), and 29 were studying Economics (ECO), with the remaining 16 enrolled in Finance (FIN). In order to guarantee the representativeness of this sample, we compared in terms of gender and degree both universe and sample for the whole population and for each degree (

Table 1. Universe vs. sample comparison. BA—Business Administration; ECO—Economics; FIN—Finance.

	Female	Χ2 (p-Value)	BA	ECO	FIN	Χ2 (p-Value)
Universe	58.09%	0.21 (0.87)	61.18%	23.03%	15.79%	0.69 (0.71)
Sample	59.24%		65.08%	23.02%	11.90%
		Female	Χ2 (p-Value)
	BA Universe	57.69%	0.00 (1.00)
	BA Sample	57.86%
	ECO Universe	62.86%	0.20 (0.66)
	ECO Sample	68.52%
	FIN Universe	54.17%	0.73 (0.39)
	FIN Sample	60.00%

^†p < 0.10; * p < 0.05; ** p < 0.01.

). Thus, it can be observed that there were no statistically significant differences, and that the sample considered can be representative. Moreover, the sampling error, ε, was equal to 3.61%, which is a good value. Additionally, 58 students from another BA course related to Business Strategy, i.e., Competitive Strategy, were considered as the study’s control group.

In order to obtain the necessary information, two self-assessment surveys were set and applied to students at two different moments during the 2017–2018 academic year.

3.2. Variables

The abovementioned surveys contained different sections, which represent the main variables of the study, with some questions that were distinct from one another (see Supplementary Materials). These variables are in line with the tension previously identified in the literature: “changing the world for the better” and “making companies better off”. Thus, students’ consumption habits (CONSUMPTION) were measured using five items created specifically for this research, from the product and label database of apps used in this teaching initiative (Abouit and Labels for your Planet, respectively). Students were asked, on the one hand, to indicate whether they were familiar with an organic product labeling as they found it when buying certain products, and, in this manner, establish what they knew about it. On the other, they were told to “choose one of the brands listed in the survey, for each of the consumer product categories proposed”. To measure students’ daily habits (LIFE), a scale was used that was made up of 13 items adapted from an initiative developed by Diputación Foral de Bizkaia [65] and the surveys carried out by Centro de Investigaciones Sociológicas [66,67], on habits linked to “energy consumption”, “water consumption”, “domestic waste management”, and “transport use”. A five-point Likert scale was applied, where 1 was “never” and 5 was “always”.

The MANAGEMENT variable (present in both surveys), about the students’ perception as to how CSR in companies should be managed, was measured using the PRESOR scale, made up of 16 items—Perceived Role of Ethics and Social Responsibility [68]. However, we should point out that some of the items on this scale were adjusted, and one was removed, after finding in the pre-tests that were carried out that it was not clearly understood; therefore, the final scale had 15 items. Students were asked to express to what extent they agreed or disagreed with a series of statements about CSR management in organizations, using a five-point Likert scale, where 1 was “totally disagree” and 5 was “totally agree”.

Before giving out the two definitive surveys, two preliminary tests were conducted with each. In the first pre-test, both surveys were assessed by a group of CSR and Business Ethics experts, to make sure that all the variables and factors under analysis were properly represented in the surveys. The second pre-test consisted of giving surveys to a group of undergraduate students to identify potential sources of error (items that were not expressed clearly, areas that were difficult to understand, inappropriate question order, etc.). This procedure resulted in the definitive version of the two surveys.

Moreover, in addition to Consumption, Life, and Management, an aggregate indicator (TOTAL) was also built to evaluate students’ socially responsible behavior, adding together all the scores for the replies in the common sections in both surveys about their habits as consumers and habits in their daily life, and the behavior of students as future managers.

In general, the criterion when tabulating the questionnaire was always the same; for those questions with only two possible replies, the one associated with a more socially responsible behavior took a value of 5, with the alternative being valued at 1. In the other questions where the reply was along a scale, this went from 1 (less responsible behavior) to 5 (more responsible behavior).

3.3. Methods

The teaching initiative took place over several sessions throughout the semester and was developed as explained below.

First session—Preliminary survey. The main aim was to discover the behavior of students, their habits as consumers in their day-to-day lives, and their behavior or perception as the managers of the future. To this end, in this initial session, students were requested to answer the first survey (Supplementary Materials) individually and anonymously. However, it is necessary to mention that a code was assigned to each student (the same code had to be used for both the preliminary survey and final survey) in order to allow us to compare outcomes from these two surveys in the results analysis phase without identifying the student. At this time, nothing was explained to students about the teaching initiative, to prevent bias in their responses and avoid skewing the results.

Second session—Imparting knowledge about sustainability and CSR using traditional methods. This second session was spent teaching basic facts and key ideas about CSR, from a dual perspective, that is, considering students not only as current consumers, but also as future company managers. The session took place within a traditional class structure, in which the teacher communicated a series of concepts and theoretical ideas, supported by images, or case studies from companies.

Third session—Imparting knowledge about sustainability and CSR using apps. In the third session, students were given information about the key CSR apps that currently exist. The idea was to inform them about the range of (free) tools available to them to take more responsible consumption decisions. At the same time, the intention was that they become aware of how certain decisions taken by companies (whether to disclose information or not to their stakeholders, to use more sustainable raw materials and components or not, to comply with labor laws or not, etc.) can affect their competitiveness and reputation. Specifically, they were shown the apps Abouit (it is a free app that provides consumers with information about the impact on society, the environment, and human health of a wide range of products and the companies that manufacture and/or distribute them; https://abouit.eu/home), Labels for your Planet (it is another free app for mobiles in which users can find out whether the products they are buying and consuming (and the company that manufacturers/sells them) have any certifications in these areas and, if they do, what that certification entails; http://www.labelsforyourplanet.com), Ekomind (http://www.ekomind.com), and Consumo Responsable Triodos (https://www.triodos.es/es/conozca-triodos-bank/que-hacemos/comunidad/consumo-responsable-triodos/), with particular emphasis placed on the first two, given that these were the apps with which they were going to interact during the subsequent session. This third session was divided into two parts. In the first one, the teachers gave students the most important information about the aforementioned apps and showed them how these apps should be used in order to help them to take more sustainable decisions. In the second part of the session, the students spent some time with the apps in order to become familiar with their content and functioning.

Fourth session—Interaction with apps: Abouit and Labels for your Planet. The fourth session reinforced the CSR skills acquired in the previous two by working actively with these two apps. On this occasion, a set of 12 products with different categories and brands were used, products that are present in students’ everyday lives and with which anyone could be familiar (toothpaste, soap, rice, detergent, ice cream, and mobile phones, among others). After splitting the class into groups of four to five students who all downloaded their apps on their mobile phones, the session was structured as described below.

• Practical assignment with the Abouit app. Each group of students was given a real product and asked to scan its barcode with the app to bring up its sustainability score, both the overall mark and in the different categories (environment, society, and health). They were also asked to justify the score for that product using the information provided by the app and to search on the app for products in the same category that had been assigned lower and higher marks, explaining the rationale behind the different scores. All this had to be written up into a report to be handed in to the teacher at the end of the session.
• Practical assignment with the Labels for your Planet app. Each group was given one or two products that had to be different from the ones they had before and was asked to use this app to identify the environmental labeling on its packaging. Specifically, they had to discover what kind of labeling it was, what it meant, what type of product it applied to, who the certification body was, that body’s geographical remit, and what other labels it had a link to, among other questions. This all had to be written up in the abovementioned report as well.
• Sharing results. Lastly, all the results were shared; for each of the products that had been used during the assignments, a spokesperson appointed by each student group explained to their classmates what they had learned in terms of sustainability, both of the product and of the company manufacturing/distributing it. Thus, with the information collated by each group and collaboratively, a fairly extensive amount of information was gathered about each product and company studied, encouraging debate and critical thinking among the students.

Fifth session—Final survey. At this final session, the students were again asked to fill in a survey (Supplementary Materials) individually and anonymously. As well as containing the same questions about their consumption habits, daily lives, and queries about their future behavior as managers, the survey also included questions to assess the teaching initiative and the type of learning acquired.

With regards to the control group students, the same two questionnaires (pre- and post-training) that were applied to students who took part in the initiative (“mixed-methods students”) were also applied to them, in two different sessions. The first one was applied at the beginning of the semester and the other one at the end of it. Additionally, two intermediate sessions were carried out with these students, although they were slightly different from sessions focused on “mixed-methods students”. That is, in this case, in these sessions, the students were given training on the same content as the rest of the students who took part into the initiative; however, only a more traditional methodology was used instead of using a combination of both traditional methodology and apps.

The information from the “mixed-methods student group” was analyzed by comparing the responses given by students in the first and second surveys (pre- and post-training), in total and by sections, in order to assess the impact of the initiative on their attitudes and intentions. Specifically, regarding the methodology employed in our descriptive analyses, we firstly checked that all the variables were distributed in a non-normal manner, which conditioned the statistics to be used in subsequent tests. Therefore, since the samples were related and the variables had a non-normal distribution, the Wilcoxon signed-rank test was used.

4. Results

Focusing on the students who took part in the initiative receiving mixed-methods teaching, it can be observed (

Table 2. Comparative analysis pre- and post-training—total sample ^[a].

	TOTAL	CONSUMPTION	LIFE	MANAGEMENT
Pre-training	113.66	11.28	46.36	56.02
Post-training	120.18	14.19	50.31	55.68
Wilcoxon	−6.25 **	−7.28 **	−5.37 **	−0.13

n = 106; ^[a] mean values are shown; ^† p < 0.10; * p < 0.05; ** p < 0.01.

) that there was a significant increase in socially responsible behavior in the aggregate variable (TOTAL) after applying the “mixed methods”, rising from an average value of 113.66 to 120.18. Likewise, the indicators for CONSUMPTION and LIFE behaved similarly, whereas, for MANAGEMENT, there were no significant variations.

Once the behavior of the sample as a whole was analyzed, we were interested in making comparisons using different classification variables. Therefore, in the first place, we analyzed students’ behavior according to the degree in which they were enrolled (

Table 3. Comparative analysis pre- and post-training by degree ^[a].

		TOTAL	CONSUMPTION	LIFE	MANAGEMENT
BA n = 69	Pre-training	111.55	11.20	44.41	55.94
	Post-training	116.94	13.90	48.41	54.64
	Wilcoxon	−4.47 **	−5.83 **	−4.43 **	−1.28
ECO n = 23	Pre-training	119.00	11.00	50.57	57.43
	Post-training	125.00	14.52	53.30	57.17
	Wilcoxon	−3.09 **	−3.89**	−1.86 †	−0.17
FIN n = 14	Pre-training	115.29	12.14	49.07	54.07
	Post-training	128.21	15.07	54.79	58.36
	Wilcoxon	−3.14 **	−1.96 †	−2.28 *	−2.26 *

n = 106; ^[a] mean values are shown; ^† p < 0.10; * p < 0.05; ** p < 0.01.

). Here, we saw that in the three degree courses, the results remained the same as before, meaning that the marks for TOTAL had risen (at a 1% level). Looking at the partial indicators, it can be seen that results differed by degree course. In the undergraduate BA degree course, there was only a significant increase in the CONSUMPTION and LIFE indices, in both cases being significant at a 1% level. In the ECO degree course, there was a significant increase in the CONSUMPTION (at a 1% level) and LIFE indicators; however, in this last case, the significance was only at a 10% level. Finally, in the FIN degree course, all three indicators rose significantly; however, the CONSUMPTION index showed significant results with a lower level (10%) than the LIFE and MANAGEMENT indexes (5% level).

Next, we compared whether there was a change in the behavior of students after the teaching initiative, taking into account demographic variables such as gender, age, and any professional experience they might have (

Table 4. Comparative analysis pre- and post-training by gender, age, and professional experience ^[a].

			TOTAL	CONSUMPTION		LIFE	MANAGEMENT
Gender	Female n = 62	Pre-training	116.10	11.52	47.31		57.27
		Post-training	122.21	14.60	50.39		57.23
		Wilcoxon	−5.28 **	−5.72 **	−4.03 **		−0.38
	Male n = 44	Pre-training	110.23	10.95	45.02		54.25
		Post-training	117.32	13.61	50.20		53.50
		Wilcoxon	−3.56 **	−4.48 **	−3.61 **		−0.62
Age	>Mean n = 32	Pre-training	110.84	10.59	44.75		55.50
		Post-training	121.56	13.59	51.38		56.59
		Wilcoxon	−4.80 **	−4.34 **	−4.24 **		−0.98
	<Mean n = 74	Pre-training	114.88	11.58	47.05		56.24
		Post-training	119.58	14.45	49.85		55.28
		Wilcoxon	−4.13 **	−5.93 **	−3.25 **		−0.88
P. Experience	Yes n = 47	Pre-training	113.21	11.02	46.04		56.15
		Post-training	121.64	14.32	51.11		56.21
		Wilcoxon	−5.08 **	−5.52 **	−4.29 **		−0.90
	No n = 59	Pre-training	114.02	11.49	46.61		55.92
		Post-training	119.02	14.08	49.68		55.25
		Wilcoxon	−3.70 **	−4.87 **	−3.15 **		−0.89

n = 106; ^[a] mean values are shown; ^† p < 0.10; * p < 0.05; ** p < 0.01.

). Sub-samples were created according to whether students were female or male, whether they had any professional experience or not, and by age. In this latter category, the average age was calculated and came to 22.42 years old, and two groups were selected: one of students younger than the average age, and another of students above the average age.

As can be seen in Table 4, the results were the same as those displayed in Table 2, independent of gender, age, and professional experience. In all the sub-groups analyzed, there was a significant increase at a 1% level in the global indicator for socially responsible behavior (TOTAL) following the “mixed method”, as well as in the partial indicators for CONSUMPTION and LIFE, with the differences being significant at a 1% level. However, there were no significant differences in any of the sub-groups in the MANAGEMENT variable.

Next, we corroborated the reliability of the results achieved using the control group, made up of undergraduates enrolled on a course with contents analogous to those of the courses where the “mixed teaching methodology” was applied. We checked for comparability of both the control group (composed of 58 students) and the group of all the students who took part in the mixed methodology, in socio-demographic terms. Thus, we observed that no significant differences were found with respect to gender, age, and work experience.

With regards to the control group, according to

Table 5. Comparative analysis pre- and post-training for the control group ^[a].

	TOTAL	CONSUMPTION	LIFE	MANAGEMENT
Pre-training	104.10	11.26	44.40	48.45
Post-training	107.14	12.22	45.24	49.67
Wilcoxon	2.13 *	2.45 *	0.74	2.19 *

n = 58; ^[a] mean values are shown; ^† p < 0.10; * p < 0.05; ** p < 0.01.

, a significant increase (at a 5% level) of socially responsible behavior by students in the TOTAL indicator was also observed comparing pre- and post-training responses. Likewise, the indicators for CONSUMPTION and MANAGEMENT behaved similarly (significant at a 5% level), whereas, for LIFE, there were no significant variations.

In this sense, even when contents relating to CSR were taught in a “traditional” way, with theoretical and practical explanations and case studies, a generally positive evolution could be detected. Thus, in order to check for the relative effectiveness of our novel teaching initiative, a means comparison of the variation rate of students’ responses from both the “mixed-methods” group and the control group was carried out (

Table 6. Comparative analysis of percentage change between Surveys 1 and 2: mixed methods vs. traditional method ^[a].

Variation Survey1 vs. Survey2	TOTAL	CONSUMPTION	LIFE	MANAGEMENT
“Mixed methods” (n = 127)	6.37%	30.71%	9.38%	0.50%
“Traditional method” (n = 58)	3.05%	10.39%	2.44%	2.91%
Mann–Whitney U	2806.50 *	2170.50 **	2899.50 *	3276.50

n = 185; ^[a] mean values are shown; ^† p < 0.10; * p < 0.05; ** p < 0.01.

). Significantly higher increases for TOTAL and LIFE indicators at a 5% level and CONSUMPTION indicator at a 1% level were observed, but this was not the case for MANAGEMENT. Thus, in general, it can be said that students who took part in our teaching initiative changed their mind to a greater extent.

Additional Analyses

In the second survey, we asked students who participated in the innovative teaching sessions to assess their learning experience. Students were asked to assess the initiative, on a five-point Likert scale, on the one hand, in terms of how much they felt they had learned during these sessions and, on the other, how much they believed they would have learned with a more traditional methodology (Supplementary Materials). The measurement scale used was made up of six items adapted from the scales proposed by McCroskey, Fayer, Richmond, Sallinen, and Barraclough [69], and by Richmond, McCroskey, Kearney, and Plax [70]. In this case, the average valuation for the first question came to 3.78, against the second, with an average valuation of 2.59. When we compared the averages for all students, we found that there was a significant difference (t = 15.49; p < 0.01) in favor of the teaching initiative focusing on mixed methods we conducted. Furthermore, the students were also asked to evaluate what was taught and their own attitude toward these contents, again on a scale from 1 to 5. Here, the evaluations were very positive for both questions, with results of 4.20 and 4.24, respectively. Finally, the students were asked the two last questions, about their interest in taking part in courses on similar subjects and using similar methodology (one for courses about CSR and another for courses using apps). In both cases, the scores were higher than 3 (3.06 and 3.30, respectively), demonstrating a moderate level of interest on the part of the students.

5. Conclusions

The main aim of this study was to evaluate the effectiveness of using mobile apps combined with traditional methods in teaching sustainability. Our results show that, in general, business students that learn with mixed teaching methods (traditional and apps) significantly improve their sustainability skills compared with the business students that learn with only a traditional method of teaching, supporting our hypothesis. Our findings contribute to the scant and non-conclusive literature in this topic [29] and to the absence of consensus about appropriate teaching methods. From this perspective, our study allows us to offer conclusions in several ways.

Firstly, our findings support the role of apps in sustainability and CSR teaching. Following the recommendations of previous authors that promoted an interactive learning style to achieve managerial competences [33,49], we show that a combination of traditional teaching with new technological devices, in particular “apps”, heighten the learning effect.

Secondly, previous studies demonstrated the efficiency of new active learning techniques from several disciplines and from a range of age groups of students [5,41,42]. However, despite the popularity of these new teaching methods that include technology as a fundamental part of the method, there is no empirical evidence about their effects on the sustainability topic [23]. This fact contrasts with the current literature on this topic and, thus, enhances the relevance of these new methodologies to teach sustainability [29,32,40]. Specifically, our study contributes to the current debate in the literature in a practical way, enhancing the positive outcome of apps in sustainability and CSR education when this method is combined with a traditional methodology. Mixed teaching methods, i.e., those that combine traditional training (textbooks, handbooks, scientific journals, etc.) with digital tools (apps, videos, other technology aids, etc.), make it possible to empower more intensive learning in sustainability issues [3,21]. From this perspective, our results suggest that it is necessary to combine both (traditional and apps) in order to modify students’ sustainable behavior to a significant and successful extent. In this sense, our students believe that using apps helped to improve their learning in sustainability more than if they had only received traditional teaching. Our findings also support previous research that analyzed similar objectives but in a sample of academics [29]. In this case, the instructors indicated “the need for more and a greater mix of teaching resources that are sustainability-focused or that integrate sustainability dimensions into different management disciplines” [29] (p. 477). Furthermore, Aragon-Correa, Marcus, Rivera, and Kenworthy [29] (p. 478) highlighted the idea that “digital resources are viewed as important and have relatively high associated levels of satisfaction for instructors in the field, even when apps appear to be in a very early stage”.

Thirdly, our findings also contribute to the current tension in sustainable education. Despite the fact that previous work defended the position that the teaching of sustainability must be focused on improving students’ CSR management skills in order to make companies better off [71,72], this study also insists on the necessity to create a better world, stating that sustainable education should also focus on this goal [40]. This paper argues that teaching sustainability and CSR should be designed not only to train managers to make companies better off, but also to teach students to change the world they live in, pursuing both goals.

As a whole, our results allow us to conclude that the initiative carried out improves students’ overall knowledge of sustainability concerns. However, our results by sections (consumption, life, and management) only show statistically significant differences in the behavior and commitment of students toward society (consumption and life), which are in line with the previous research [29]. One reason why the differences in CSR management are not statistically significant may be the absence of analysis into the long-term effects of this type of learning. This is a limitation of this research. While the teaching has an almost instantaneous effect on consumption, the effects on management require more time. This question constitutes a line of research for the future. Additionally, another explanation could be that the impossibility of finding apps more focused on CSR management limits the capacity for teaching on this subject, since perhaps the apps used have a greater impact on consumer lifestyle and behavior. Another limitation may be that the sample size for this study is relatively small, as it is focused on only one Spanish university. Thus, it would be useful to widen its scope of application, targeting other universities and samples. With regards to the methodology, although we carried out an analysis with the aim of adding empirical evidence to the strand of the literature, only a descriptive analysis was conducted. In the future, regression analysis might be considered for the purpose of studying the influence of combined teaching methods on students’ commitment to CSR and sustainability.

Despite the limitations presented above, we can conclude that the combinations of traditional and action learning methodology, and in particular apps, in higher education are widely beneficial for business sustainability and CSR topics. Our results can empower teachers to make flexible use of the affordances to design learning activities for the purpose of achieving intended learning outcomes across different settings. It can also empower teachers to use apps for assessing students’ learning processes and identifying their learning problems to make pedagogical refinements where necessary, as we argued in this paper.