Education for Sustainable Development 4.0: Lessons Learned from the University of Graz, Austria

Abstract

Higher education for sustainable development plays a crucial role in the transformation of society towards a more sustainable pathway of development. The new trends in society and technology experienced in the course of the fourth industrial revolution come with challenges, but also provide opportunities. In this paper, we reflect on the conceptual basis of education for sustainable development as approached at the University of Graz, Austria, and contrast this basis with the expectations stated by students. The results showed that students acknowledged the high importance of digital competencies and found it highly important to be confronted with future-oriented topics and contents. Interestingly, students seemed skeptical about online course formats and digitalization of teaching and clearly preferred the interactive classroom experience. Students also rated international topics and transdisciplinary thinking as fairly important. Interestingly, a relatively high share of students only saw mediocre added value in experiencing international classrooms, and in having international teachers, when it comes to meeting their top priority, which is being competitive in the job market. Education for sustainable development in the future will not only need to prepare students for international, interdisciplinary, and digital environments, but also will need to meet the expectations of demanding and ambitious students and provide them with bright career prospects.

1. Introduction

Higher education for sustainable development addresses ill-defined, highly-complex real-world problems, such as climate change, pollution of environmental media, exhaustion of resources, overproduction of phosphorus and nitrogen, biodiversity loss, or unjust distribution of wealth. The relevance of the topic is recognized within the United Nations Sustainable Development Goals (SDGs). As a part of SDG 4 (quality education), Target 4.7 states “that all learners acquire the knowledge and skills needed to promote sustainable development, including, among others, through education for sustainable development and sustainable lifestyles, human rights, gender equality, promotion of a culture of peace and non-violence, global citizenship and appreciation of cultural diversity and of culture’s contribution to sustainable development” [1]. In general, there is a broad understanding that higher education for sustainable development plays a crucial role in the transformation of society towards a more sustainable pathway of development [2,3,4]. Thus, higher education for sustainable development needs to be transformative in the sense of challenging worldviews, assumptions, and values we as a society hold [5].

The transformation of any kind of system, such as a region, an enterprise, or any other entity, towards a more sustainable pathway requires an understanding of highly-interlinked social, economic, and environmental factors that determine the addressed real-world problems. Hence, it has been argued that for a profound system understanding, the integration of different kinds of knowledge and modes of thinking, and thus an interdisciplinary orientation of study programs is urgently needed. Bringing together competences from different fields in an interdisciplinary way is necessary to integrate different paradigms and methodologies. This integration facilitates cross-disciplinary thinking that precedes the development of transformative competencies and commitment, which contribute to solving complex real-world problems in ways that would have been unlikely through single disciplinary means [6]. Diverse student teams proved to incorporate a wider problem context in problem-solution propositions [7].

However, interdisciplinarity in the sense of combining analytical knowledge from different academic disciplines is not yet sufficient for tackling real-world problems. Instead, it is also important to bring together analytical knowledge and system understanding with practical experiences and perceptions, and linking these to society’s values and preferences, so that a joint problem-solving process amongst science and society can be achieved [8,9,10,11,12]. Scientific expertise and practical experience are different sources of knowledge, but of equal value. Mutual learning [8] between academics and practitioners can result in an integrated production of knowledge that is socially robust [13]. Through its “epistemological pluralism” [14], transdisciplinarity brings reflexivity into the processes of knowledge production [15], so that students are supported and encouraged in their development towards being critical and reflective thinkers and problem solvers. Similarly, Mader [16] identified education and learning, participation, and research integration as three out of five main principles in the “Graz Model for Integrative Development”.

In short, there is vast agreement that sustainability sciences and studies on sustainable development ultimately require inter- and trans-disciplinarity. The complex and ill-defined problems sustainability sciences are facing cannot be addressed sufficiently from a mono-disciplinary perspective. Thus, respective study programs need to consider these observations in order to unfold the potential of (higher) education for sustainable development (ESD).

Inter- and trans-disciplinary thinking are elementary, but not the only basis, in ESD. Sustainable development topics cannot be sufficiently covered on national levels, but require international cooperation; including the international perspective is crucial for ESD, and it has been argued that it is only obvious to also organize ESD through international cooperation between higher education institutions [17]. Internationalization clearly is one of the key dimensions of higher education for sustainable development. This is reflected in the SDGs, in particular in Target 4.C of SDG 4 (“Quality Education”) [1], but also in the reports on the “United Nations Decade of Education for Sustainable Development” from 2005–2014 [18]. At the same time, education for sustainable development also needs to address the recent trends in society and technology that strongly affect the way we learn, teach, and understand knowledge and education [19]. This on the one hand involves the integration of the respective topics and methods in curricula and on the other hand the application of innovative learning technologies such as massive open online courses (MOOCs) and e-learning.

The aim of this paper is to reflect on the conceptual basis of education for sustainable development at the University of Graz, Austria, where the first respective study programs were established approximately 20 years ago. Following an international and interdisciplinary education approach involves challenges for students and teachers. Therefore, we also report the results of a survey among students in sustainable development and related fields (n = 167) and contrast their views and expectations with the elements of education for sustainable development.

The paper is structured as follows: In the following Section 2, we present the case of ESD at the University of Graz and detail its main pillars, namely inter- and trans-disciplinarity, internationality, as well as digitalization and complex systems sciences. In Section 3, we describe the methods used for the quantitative part of this study, and in Section 4, we discuss the importance of these elements from the perspective of students. Section 5 provides a comprehensive discussion and concludes the paper.

2. Elements of ESD: The Case of the University of Graz

In this section, we discuss on a conceptual level how the complexity of ESD can be addressed and currently is being addressed in the relevant curricula of the University of Graz, Austria. With its over 32,000 students and around 4300 employees, the University of Graz is the second largest university in Austria; it consists of six faculties, and the first interdisciplinary curriculum targeting the interactions of human-made systems and the natural environment was introduced in the late 1990s as an irregular study program. The curriculum was later transferred into regular programs, and ESD is now an integral part of the curricula of “Environmental Systems Sciences”. At the bachelor’s level, students take courses on complex systems sciences and in addition choose one of four disciplinary specializations, all with a focus on sustainable development. At the master’s level, students may choose from six different master’s programs, such as sustainability-oriented management, climate change, and environmental technology, but also programs based on geography or economics. Moreover, there are two joint programs with international partner universities, namely the Joint Master’s Programme in Sustainable Development and the Joint Master’s Programme in Circular Economy. The cornerstones of these programs are (1) inter- and trans-disciplinarity, including student-engagement in teaching, (2) an international perspective, and (3) a strong focus on complex systems sciences, including topics like digitalization, artificial intelligence, and big data. Figure 1 outlines the integrative approach consisting of these three elements. A detailed presentation of the elements is provided in Section 2.1, Section 2.2 and Section 2.3.

2.1. Interdisciplinarity, Transdisciplinarity, and Student-Engagement

In the case of the curricula of the undergraduate and graduate study programs in the field of Environmental System Sciences, the University of Graz follows a double strategy concerning inter- and trans-disciplinarity: On the one hand, students build up disciplinary expertise in a selected major field, which can be business management, economics, geography, or natural sciences and engineering. The underlying conviction is that profound single disciplinary expertise is a prerequisite for any interdisciplinary discourse. On the other hand, students synthesize these disciplinary competences in modules and courses, where students with different major fields and teachers from different disciplines work together in interdisciplinary teams on sustainability-related real-life problems. Whenever possible and reasonable, practitioners are integrated as co-teachers in these classes. The prime example of an inter- and trans-disciplinary educational setting is the so-called “Interdisciplinary Practical Exercise” (IP). Each study year, the University of Graz offers ten to fifteen IPs, each of them addressing a different real-world problem. The selection of the topics is mainly up to the students, since it is the students who bring up the ideas for new IPs in the planning phase. The students also contact professors from different disciplines and practicing experts and ask them if they would be willing to engage in the IPs in the upcoming study year. In this way, a planning group, consisting of professors, practitioners, and students, is created. The planning group conceptualizes the IP by defining the learning and/or research objectives, the scientific methods that will be applied, as well as the didactical concept. The student members of the planning group are incentivized by getting a guaranteed place in the planned IP. However, before the IP is actually held, it must pass a hurdle: a selection committee, which is a sub-committee of the curricula committee, evaluates the proposed IPs according to a given scheme and selects the best ones. Only these are then actually included in the university’s teaching program of the next study year. IPs at the bachelor’s level usually have a stronger focus on the application of knowledge in practical settings, while IPs at the master’s level are usually more research oriented and, in most cases, also held in English in order to integrate international students. Thus, the idea of an inverted classroom [20] is taken one step further, as students do not only participate actively in class, but already are engaged in the planning phase of the course.

The didactical concept of these IPs is a “pull-concept of learning” [21] and is characterized by the strong intrinsic motivation of students. The students are facing situations where they need to apply knowledge and competences, which they might not yet entirely have. In this way, students are being motivated to gather additional competences for their work in real-world situations, e.g., for the interaction with different stakeholder groups. In this way, they can always interlink theoretical knowledge from different sources with the practical application in the real-world system. The professors have the responsibility to set up the organizational framework of the IP work in a way that leads to an “optimal level of problems”. Too high a level would lead to potential failures, inducing frustration and demotivation, whereas too low a level would result in a decreased demand for additional knowledge and hence undermine students’ motivation to learn. Student learning within the IP is largely self-regulated. Students are pushed to recognize the consequences of their own actions and may no longer attribute success or failure to forces beyond their control. This self-regulated learning can itself be viewed as socially sustainable in the sense that it empowers students to take learning into their own hands [22]. The learning involves the whole spectrum of activity, from analysis and synthesis to formulation of conclusions, as well as the development of critical thinking and social competences.

The didactic concept leads to intense learning not only for students, but to mutual learning for all, including professors. Inter- and trans-disciplinary interaction forces all participants to be continuously open for self-reflection on what they are doing and what they are claiming. Professors from different disciplines recognize that complex and uncertain real-world systems cannot easily be managed top-down, that they alone are no longer in a position to identify a single correct way of living, or the “best form of sustainability” [23]. Instead, they rather need to understand how to move from societal needs and preferences towards the provision of generally-acceptable solutions and technologies [10].

2.2. International Topics, Students, and Teachers

For single national universities, it can be difficult to cover the aspect of internationality sufficiently, and the University of Graz is no exception here. While international topics can be integrated in curricula quite easily, for example through respective reading material or case studies in international contexts, this is not the case for integrating international students and teachers. International students might face visa issues, and teachers in addition may have difficulties in obtaining work permits; moreover, students and staff usually have to comply with the thresholds of acknowledged English language tests such as TOEFL and IELTS. The costs for such certificates and the fact that they expire after only two years is a major obstacle especially for students from developing countries; high living costs in Western countries and limited availability of scholarships are additional issues.

Education on a cross-sectoral topic like sustainable development however requires international cooperation of higher education institutions (HEIs). Such a collaboration can bundle the strengths, workforce, and expertise of different universities. In collaboration, universities may be able to offer programs of higher quality than single universities could individually. A frequently-used approach to implement such a cooperation is the creation of international double or joint degree programs. The University of Graz has implemented two such joint master’s programs related to sustainable development together with international partners (i.e., the Joint International Master’s Programme in Sustainable Development (see www.jointdegree.eu/sd/ (accessed 10 April 2019)) and the Erasmus Mundus Master’s Programme on Circular Economy (see www.jointdegree.eu/circle/ (accessed 10 April 2019)).

In this context, the European Union (EU) gained substantial experience with its “Erasmus Mundus Joint Master’s Degrees” (referred to as “Erasmus Mundus Joint Master’s Courses” before 2014). These are traditionally “seen as a means to integrate and internationalize curricula, develop strategic international academic collaboration and provide a unique study experience to students who are keen to learn from different education systems” and “could become a means to advance study recognition, quality and mobility across borders and enhance graduate employability” thorough the Bologna Process [24]. This specific approach of international teaching “contributed to significant transfers of knowledge for overcoming disciplinary administrative or even legal boundaries and promoting participative teaching, evaluation and recognition practices” [25]. With such a joint degree program, benefits can be created in a manifold way, in particular for (a) students, (b) scholars, and (c) the HEIs involved:

(a)

Students joining an internationally-organized bachelor’s or master’s program experience different learning environments and profit from the SD related knowledge from different universities, their individual strengths, and their various perspectives [17]. Moreover, students with different citizenships, jointly attending classes, provide their classmates with new background information, but also case studies (e.g., SD problems in industrialized versus emerging economies) or data sources and potential solution strategies [26]. Thus, local and international students bring in and receive benefits in a mutual way. The acquisition of intercultural competencies was ranked first (and in front of career- and subject-related expertise) when asking Erasmus Mundus graduates for the greatest impact of their joint program [27].

(b)

Scholars, respectively teachers, profit from such an international classroom, as well, as the discussions will be usually broader and more vivid. Their non-local students deliver new insights and support the increase of the variety of topics and contexts in the sustainability discussion.

(c)

HEIs, which joining forces in order to establish such a double or joint degree program in SD, can attract new students and extend their network of colleagues in similar fields, which might lead to a future co-operation in research. Furthermore, such an international program increases the visibility and the image of the participating universities.

Beyond that, joint degree programs develop innovative and/or adequate methods of teaching and knowledge exchange for their students, who are usually from different continents and with a range of different backgrounds, mentalities, and learning approaches. These methods can support and strengthen active engagement and responsible behavior in their future professional career.

However, such double or multiple degree programs with joint curricula can face several problems: one of these problems is mentioned in a global analysis of 78 postsecondary educational programs in “Industrial Ecology” (IE), which is an important and emerging sustainability-oriented multidisciplinary field. The analysis concluded that “most IE programs and courses being offered are at the graduate level from institutions within northern occidental”. Moreover, there is a requirement for a “collaborative approach when it comes to curriculum development and implementation” to deal with the multidisciplinary nature of IE countries [28]. A similar problem is the missing balance in the origin countries of guest teachers that are invited for joint programs: an analysis of 1400 scholars, which received an Erasmus Mundus scholarship between 2004 and 2010 for contributing to 57 joint degree programs, highlighted that 27% of all non-European scholars were from North America. To cope with this weakness, joint programs should try to achieve a fair balance of countries and continents when inviting guest scholars. This is also one of the key conclusions of recommendations of a report on joint international master’s programs of the EU [25]: “The best results were achieved when they [the visiting scholars and professionals] met all the partners and were involved in developing joint research and opening the network for extension beyond Europe”. In a similar vein, it is advisable to be aware of a potential “Western” or “Eurocentric” focus of the provided teaching and learning materials [29].

A recent report on experiences with Erasmus Mundus Joint Master’s Degrees dealt with further challenges of international mobility from a student’s perspective. It listed several issues, including finding accommodations, working through visa processes, and learning new languages, which all add to the foundational challenge of adjusting to a new culture [24] (p. 28). Interestingly, a paper from an Indian perspective found similar issues, when analyzing the growing number of international students in Chennai. Besides English language problems, the authors particularly mentioned funding and visa issues as significant barriers [30].

2.3. Complex Systems Sciences and Digitalization

The third main element of ESD at the University of Graz is a strong focus on complex systems sciences, including current and upcoming topics around big data, artificial intelligence, and digitalization. Systems sciences research at the University of Graz has a strong focus on quantitative methods. These include equation- and agent-based modeling, for instance of urban traffic systems or of cooperative network interactions, data mining and text extraction from unconventional sources, but also advanced statistical ways of forecasting sudden regime shifts in feedback-driven systems, as may occur in financial speculation bubbles or in climate conditions through global warming. One of the backgrounds of this focus is the belief that the complexity of many of today’s problems needs to be tackled in a new way. Conventional component analysis that hopes to find relevant insights by scrutinizing isolated parts under laboratory conditions cannot be expected to yield answers for phenomena that should be seen as emergent in the sense of being “more” than the sum of their parts. Such phenomena necessitate a “new kind of science” [31]. They need an approach that counts on dynamic non-linear causal dependencies and tries to investigate them with the only tool that is able to account for these complexities appropriately: the digital computer.

2.3.1. Challenges in Teaching Complex Systems Sciences

Complex systems sciences are virtually the methodological backbone in the sustainable development programs. The communication of respective contents however is a complex endeavor in several respects: one of them is the fact that sustainable development topics are broadly diversified per se. The respective curricula at the University of Graz, e.g., comprise comprehensive knowledge domains including business topics like corporate sustainability management, adoption of energy-efficient technologies, life cycle assessments (LCA), but also urban and regional development or natural science topics such as atmospheric dynamics or mountain hydrology, to name just a few examples. Sustainable development education therefore requires a substantial methodical apparatus able to cover diverse needs of applied and fundamental investigations.

The diversity of topics is also followed by diverse groups of students who, especially at the master’s level, come from different disciplines and have different methodological knowledge. They thus ask for a method set that on the one hand is comprehensible and on the other hand flexible concerning applications to different domains. Here, an additional challenge is induced by the joint programs offered, as students who study at the University of Graz only for one or two semesters have a limited time to grasp the methodological basics of systems sciences and still want to benefit from the advanced courses.

With regards to teaching contents, the demand for methodological comprehensiveness is met by an ever-growing pool of advanced statistical, mathematical, and computational methods and tools. They are based on classical approaches such as causal loop analysis of cybernetic feedback control or system dynamics modelling of global resource and consumption relations. Meanwhile, they are rapidly complemented and expanded from insights of investigations such as in the resilience and stability conditions of dynamical systems, the attraction of complex equilibria states, the control of unstable behaviors, the forecast and anticipation of critical transitions or tipping points, the loss of cooperation in networked social interactions, or the optimization of complex multi-user supply networks, to name just a few. All these vividly developing fields are further enriched by a rapid increase in the amount of relatively easily-accessible and thus scientifically-usable data repositories and thus by a growing demand to supply and mediate additionally state-of-the-art knowledge about contemporary data analysis and machine learning methods.

2.3.2. Digitalization of Teaching

On the teaching side, one way to provide this comprehensive methodological toolkit and to meet the challenges outlined above is the intensive use of digital means. This includes not only the digital provision of teaching materials, but also conveying of digital competencies.

In view of the rapid technical development and the strongly-increasing demand for respective skills on the labor markets, higher education institutions need to react and adjust their curricula. At the University of Graz, the bachelor’s curriculums related to sustainable development and environmental systems sciences were refurbished in 2017 to implement a basic introduction to scientific programming (with Python) as a compulsory introductory course for all students. From the first day of their studies, students now are confronted with digital scientific methods and their possibilities. Students are also made aware of the implications and possible consequences of these research methods. One example is that artificial neural networks today are capable of cognitive accomplishments that are not always entirely transparent, even not to those who developed these networks.

These methods and their implications are not always easily communicated, in particular not in the form of conventional linearly-written textbooks. Digitalization of teaching tasks seems only obvious when it comes to conveying digital competencies; besides standard e-learning methods, two more innovative forms are currently being used at the University of Graz.

The first one makes intensive use of the Jupyter Notebook IPython (https://ipython.org), an interactive platform for the scientific use of the programming language Python. This platform provides the possibility of writing and editing explanative texts, executing computer code, and visualizing results in one and the same format (for an example, see http://systems-sciences.uni-graz.at/etextbook/bigdata/simple_stats.html). The lecture notes produced in this medium are provided to students via an e-learning platform, which offers the additional possibility to assign exercises and examination tasks to students. These lecture notes offer the chance to learn complex matters in a vivid way and test them at the same time by applying variations or even performing small experiments. The possibility to execute the code immediately when reading its explanation has turned out to be an extremely fruitful didactic method. The feedback from students is very encouraging and gives reason to further expand the deployment of this training method.

The second form of digital education to mediate complex content is an online textbook with interactive elements. This e-textbook (systems-sciences.uni-graz.at/etextbook) builds on the JavaScript platform jQuery (https://jquery.com) to make its content transferable between different browsers and devices. It implements MathJax (https://www.mathjax.org) for a correct display of mathematical formulas and, most importantly, smoothly integrates with tools like Processing (https://processing.org) to generate interactive online applications of system scientific models. Content-wise, the e-textbook is structured in a non-linear way to do justice to the interconnectedness of most of its topics. The structure suggests a sort of circular perusal, bringing into focus certain topics repeatedly, but from changing perspectives, thus forcing one to not finally tick off a learned content once and for all. This interconnectedness is attempted to be expressed via a dynamic table of contents at the start page of the e-textbook.

3. Methods

While Section 2 of this paper focused on the conceptual basis of sustainable development education at the University of Graz, i.e., the supply side, it is also important to consider the demand side, i.e., the expectations of students who study in the respective programs. Therefore, we conducted a quantitative survey among students in environmental systems sciences and sustainable development programs at the University of Graz at the bachelor’s and master’s level. The survey aimed at grasping the students’ perspective on the teaching concept, and its results are presented in Section 4. The survey was carried out as an online survey in June 2018 using the tool LimeSurvey. All students of relevant programs received an invitation to participate in the survey and were asked to rank different factors/elements/characteristics of their study program regarding importance (how important is the respective factor to them). In addition, open-ended questions allowed gathering more detailed information and explanations. One hundred sixty five students participated in the survey; 111 (45 male, 66 female) of them were at the bachelor’s and 54 (32 male, 21 female) at the master’ level (of which, 15 studied in an international joint program). The mean age of bachelor’s level students was 22.7, and the mean age of the master’s level students was 25.9. The sample overall was a good reflection of the student population in the surveyed programs. In total, 27 items were used to collect student opinions; 14 items included rather general aspects of the study program, such as contents and didactical features; two items referred to the preparation for a future academic or other professional career; five items focused on the acquisition of different types of competencies; and six items aimed to grasp the importance of different course formats. Respondents were asked to rate the importance of all these items on a 5-level Likert scale (1 = very important/5 = very unimportant). Data were analyzed by simple descriptive statistics: non-parametric Mann–Whitney U tests were applied to test for differences between groups (bachelor’s vs. master’s students; students in a local master’s program vs. students in a joint international master’s program). In addition, students were asked to provide answers to open format questions regarding strengths/weaknesses, respectively advantages/disadvantages, of their particular study program.

One limitation of this survey was that it only included active students, but not former students or alumni. Another limitation was a possible non-response bias; the survey was sent to all active students, but it was not clear whether the views of non-responding students significantly differed from the responding students. Given the acceptable number of respondents, we however assumed that the sample of respondents sufficiently reflected the population of students in the sustainability-related study programs at the University of Graz.

4. Importance of ESD Elements from the Students’ Perspectives

4.1. Survey Results

The results of the survey indicated that the basic elements of sustainable development education (inter- and trans-disciplinarity, internationality, complex systems, and digitalization) were not necessarily those elements that were considered the most important ones by the students for their academic education.

Table 1. Importance of different study program characteristics, competencies, and course formats. Highest importance scores per category are highlighted in bold.

	Total (n = 164)		Bachelor’s (n = 111)		Master’s (n = 53)
General Factors *	Mean	SD	Mean	SD	Mean	SD
Expertise of teachers	1.29	0.49	1.25	0.49	1.36	0.48
Future-oriented topics and contents	1.33	0.58	1.27	0.52	1.45	0.67
Preparation for the job market	1.39	0.61	1.39	0.59	1.40	0.66
Didactic skills of teachers	1.40	0.65	1.41	0.65	1.38	0.66
Provision of teaching materials and further information	1.52	0.74	1.42	0.73	1.72	0.72
Flexible selection of interesting courses and topics	1.56	0.77	1.59	0.81	1.51	0.67
Depth of contents/options for specialization	1.61	0.67	1.72	0.72	1.38	0.49
International topics	1.74	0.96	1.78	1.00	1.64	0.88
Broad range of contents	2.00	0.88	2.01	0.83	1.98	0.99
Different disciplines/inter- and trans-disciplinarity	2.10	0.88	2.18	0.94	1.92	0.73
Remote studying (video lectures, blended learning, etc.)	2.39	1.29	2.20	1.26	2.79	1.28
Preparation for a scientific career (PhD)	2.51	1.15	2.50	1.12	2.53	1.23
Teaching in German	2.59	1.37	2.27	1.21	3.25	1.47
Teaching in English	2.61	1.28	2.81	1.30	2.19	1.16
International teachers	2.82	1.21	2.92	1.21	2.62	1.18
International students in classroom	3.28	1.35	3.47	1.28	2.89	1.44
Competencies *	Mean	SD	Mean	SD	Mean	SD
Content knowledge/content competencies	1.31	0.51	1.36	0.54	1.21	0.45
Method competencies	1.63	0.81	1.59	0.79	1.70	0.85
Personal development/personal competencies	1.65	0.89	1.67	0.95	1.62	0.77
Digital competencies	1.69	0.87	1.74	0.92	1.58	0.75
Social competencies	1.91	0.99	2.01	0.99	1.72	0.97
Course Formats *	Mean	SD	Mean	SD	Mean	SD
Interactive courses	1.88	0.98	1.95	1.01	1.74	0.90
Excursions	2.07	1.21	2.10	1.20	2.02	1.23
Group work courses	2.15	1.14	2.14	1.10	2.19	1.24
Seminars	2.23	1.01	2.23	0.99	2.21	1.04
Lectures	2.37	1.12	2.20	1.05	2.72	1.18
Online courses	2.57	1.28	2.32	1.21	3.11	1.27

* 5-level Likert scale; 1, very important; 5, not important at all.

shows the results of the importance ratings done by bachelor’s and master’s students on five-level Likert scales (1 = very important, 5 = not important at all). Among bachelor’s students, factors like the expertise of teachers, future-oriented contents, preparation for a professional career, the didactic skills of teachers, and the quality of provided learning materials received very high importance ratings between 1.2 and 1.5 on a five-level Likert scale. The situation was similar with master’s students; they however rated the importance of learning materials slightly lower than bachelor’s students (1.72 vs. 1.42, p = 0.003) and the importance of content depth and specialization options slightly higher than bachelor’s students (1.38 vs. 1.72, p = 0.004). Inter- and trans-disciplinarity were considered important, but relatively less important than other factors (mean 2.1 on the five-level scale). The language of instruction seemed not to matter much, as the respondents neither considered teaching in English or in German (the native language of most students) as particularly important. Options for remote studying were not considered a factor of major importance, but appeared more important to bachelor’s students than to master’s students (2.20 vs. 2.79, p = 0.004).

Interestingly, the importance of internationality received mixed ratings. While both bachelor’s and master’s students stated a fairly high importance of international topics in their study program, having international teachers was only considered of medium importance. Sharing the classroom with international students was considered relatively unimportant by bachelor’s students, and also not very important by master’s students. In fact, these two items—international students and international teachers—received the lowest importance rankings among all factors.

The different types of competencies that may be acquired through a study program all received fairly high importance ratings. The most important competency clearly was content knowledge; for bachelor’s students, method competencies came in second, and for master’s students, digital competencies were the second-most important one. Social competencies ranked lowest among the different types (but still received a fairly high rating); the differences here however were relatively small and did not allow for deriving major implications. When comparing bachelor’s and master’s level, it turned out that social competencies were considered more important by master’s students (1.72 vs. 2.01, p = 0.039).

There were no major differences regarding the importance of different course formats for bachelor’s students; all surveyed formats received similar importance ratings from them (mean ranging from 1.95 for interactive courses to 2.32 for online courses). Master’s students are clearly distinguished here; for them, interactive courses were most important (mean 1.74), while lectures (2.72) and online courses (3.11) received relatively low importance ratings. The latter two importance ratings exhibited significant differences between bachelor’s and master’s students (lectures 2.20 vs. 2.72, p = 0.009; online courses 2.32 vs. 3.11, p = 0.000).

While bachelor’s programs at the University of Graz are predominantly taught in the national language of German and mainly attract local students and students with German as their native language, the master’s programs related to sustainable development are predominantly taught in English and also attract international degree-seeking and exchange students. Especially in the international joint master’s programs, which are entirely taught in English, there is a high share of international students. We therefore also compared the answers of master’s students in local programs (n = 38) with the answers of master’s students in international programs (n = 15). In only five of the 27 survey items we found significant differences between these two groups; however, due to the small number of respondents, these differences only illustrate a tendency and shall be interpreted with care: flexibility with regards to course selection and specialization options was more important to students in international programs (1.20 vs. 1.63, p = 0.028); lectures as a course format were more important to students in the international programs (1.80 vs. 3.08, p = 0.000); with regards to language, teaching in English was obviously more important to the international students than to the local students (1.40 vs. 2.50, p = 0.001), and this preference reversed for teaching in German (4.07 vs. 2.92, p = 0.007); here, it can be observed that teaching in German was not seen as important in general, not even by the master’s students in local programs. One interesting observation, especially when considering the “internationalization at home” strategies promoted at the European level and implemented by HEIS, is how students perceived international classrooms. While the international classroom experience, with a high share of international students in class, was very important to the students in international programs, this seemed not to be the case for students in local programs (1.60 vs. 3.39, p = 0.000). The international and local students in the sustainable development programs shared several courses with each other, and therefore, both experienced an international classroom. However, the perception of how important such an experience is differed significantly.

The findings from open format questions in the survey underline the quantitative results. The respondents at the bachelor’s and master’s level predominantly rose one major theme, which was a strong desire for practice orientation in the courses and for preparation for the job market. Bachelor’s students who planned to continue with a master’s program stated that they “did not yet feel ready for the job market”. While some respondents criticized that the covered contents were too broad and sometimes lacked specialization options, others appreciated the broad opportunities regarding a future career and the relevance of the topics in the program.

Comments regarding international students in the classroom were mixed. Students in the joint master’s program appreciated the international setting, the interaction with students of different cultural backgrounds, and the mobility options within their programs. Mixing international students and local students definitely has benefits; it however also comes with challenges. One big challenge is the different backgrounds and different levels of pre-knowledge. While local master’s students went through several courses in complex systems sciences, international students sometimes lack this previous knowledge; this is a challenge for teachers who need to make sure that all participants can follow and understand the inputs, but also a challenge for student groups, when, e.g., in single cases, the more knowledgeable students feel that those with less pre-knowledge are free-riding in group assignments.

4.2. Insights from Access Statistics of Online Material

Moreover, detailed access statistics on the open access online learning materials indicated which topics in the branch of complex systems sciences were the most popular ones. Note that, due to the open access to these materials, these statistics do not necessarily reflect the interests and learning assignments of students in Graz, as a large part of textbook users seemed not to be attending a study program in Graz.

With an opt-out option, we implemented the analytics software Matomo (matomo.org, formerly known as piwik) in order to provide information about several aspects of the use and acceptance of the e-textbook. These statistics revealed for instance that, despite its rather specialized contents, the e-textbook had been visited by citizens from all over the world, apart from a few countries in Africa.

Since the e-textbook is used to support teaching at the University of Graz, the by far largest share of users accessed the textbook from Austria. Apart from the two foci points in Graz and the capital Vienna, users were spread over the entire country. The largest user groups from outside Austria came from other Western European countries (16.1%) and Northern America (15.8%); 6.1% of users accessed the textbook from Asia. Users from Middle and South America, Russia, Turkey, and Australia were a minority among the users (

Table 2. Origin of visitors of the e-textbook for systems sciences (according to IP addresses).

Country	Visits
Austria	56.0%
Western Europe	16.1%
North America	15.8%
Asia	6.1%
Middle/South America	2.4%
Russia/Turkey	2.0%
Other/Unknown	1.6%

). The e-textbook covers a variety of topics;

Table 3. Top 15 popular topics in the e-textbook for systems sciences.

Rank	Topic	Hits
1	Stability analysis with Python	10,877
2	Examples of social dilemmas	6700
3	Phase plane analysis	6314
4	SIR-A Model for Epidemiology	5274
5	SIR in various platforms	4855
6	System dynamics	4509
7	Cellular automata	4458
8	What is a system?	4591
9	Attractors	4111
10	Lyapunov	3604
11	Equation-based versus Agent-based Modeling	3580
12	Game of life	3718
13	Critical transitions	3547
14	Public goods game	2927
15	Early based warning signals with Python	2955

lists the 15 most frequently-accessed topics.

5. Discussion and Conclusions

Education for Sustainable Development at the University of Graz is built on the three pillars inter- and trans-disciplinarity, internationality, and complex systems sciences and explicitly includes new developments over the course of the fourth industrial revolution. The importance of this element was also recognized by students, who ranked future-oriented topics and contents as some of the most important elements in their study programs. At the same time, students acknowledged the importance of digital competencies; the survey reported in this study clearly indicated that especially master’s students considered digital competencies as central in their education.

As outlined in the call for papers to this Special Issue [19], digitalization is one of today’s key trends and is affecting all societal areas, including (higher) education. When it comes to the digitalization of teaching, students at the University of Graz however were not comprehensively positive. MOOCs or online courses did not receive high importance ratings in the survey; especially master’s students saw the classroom experience in interactive courses as most important. Traditional-style lectures received lower importance ratings, and online courses even held the “lanterne rouge” and took the last place in the importance rankings. As a side note, bachelor’s students seemed more open to online courses than master’s students, who preferred the interactive classroom experience.

This observation might also relate to a status quo bias among students who are not yet used to the full range of possibilities and opportunities that, e.g., MOOCs or webinars are offering, but only are familiar with online components that are complementary to offline courses. It might also be worthwhile to combine virtual learning environments with inverse classroom concepts. In the context of ESD, jointly-developed webinars—done by students and instructors from different countries—could create new contents, facilitate international discussions, and at the same time reduce the need for travelling. Engaging students in the development of digital teaching means might also mitigate their current skepticism. Furthermore, virtual teaching assistants, e.g., in the style of Amazon’s intelligent personal assistant “Alexa”, could revolutionize higher education in general, and ESD in particular. In any case, it can be stated that digitalization in higher education comes with pros and cons, and its application has to be carefully determined in terms of quality, quantity, and intensity.

Another interesting observation is that, while international topics were considered as highly important by most SD students, international teachers and international students in the classroom were not. The international classroom experience was important mainly to those who register for international programs and joint programs; it however was not relevant to those who study in a national mono-degree. This result however has to be interpreted with care and might be specific to the case of Graz; one explanation, e.g., could be that international students with different pre-knowledge on methods from complex systems sciences were perceived as free-riders in group assignments by some of the national students, while others appreciated the additional perspective they brought to the classroom. The relatively high standard deviation for this item suggests a polarization of perceptions in this regard.

An important factor that should not be forgotten in ESD is the prime expectation students have for a study program: they expect to get sufficiently prepared to compete successfully in the job markets. As the job profile of SD graduates is much less clear than, e.g., the profiles of graduates of a law school or medical school, this requirement from the student side is understandable. One particular challenge for sustainable development education therefore is to balance the conflicting priorities of conveying a broad set of topics on the one hand and job training on the other hand. The living realities of students and their expectations must not be ignored when designing curricula along the three pillars discussed; if (potential) students do not perceive how an interdisciplinary and international education benefits their career prospects, they most likely will not sign up for respective programs, or eventually drop out. While a specific job training is not within the responsibilities of universities, at least not in Austria, it still needs to be made sure that students, and ultimately also employers, recognize the assets that graduates of sustainable development programs can bring to companies. One obvious way to achieve that is close cooperation with industrial partners who may provide possibilities for internships to students of SD programs. The newly-developed Erasmus Mundus Master’s Programme on Circular Economy comes with such an option, and it is expected that students in the program will receive this option favorably.

ESD in the future will not only need to prepare students for international, interdisciplinary, and digital environments, but also will need to meet the expectations of demanding and ambitious students and provide them with bright career prospects.