A Preliminary Model for Assessing University Sustainability from the Student Perspective

Abstract

This paper assesses university sustainability from the perspective of the interested student. A set of questions for a university website analysis is proposed and preliminary results for Swedish universities are presented. The university website analysis intends to emulate a student looking for a university working with sustainable development. University ranking is compared with the results from the sustainability assessment.Results from the study are based on university website analysis of 18 Swedish universities out of a total of 30. Universities are grouped in high ranked, low ranked and benchmark universities. For the majority of the studied universities it was possible to extract the information needed for a sustainability assessment from the website, which indicates that further development of the method is of interest. The average level of performance in the assessment was found to be less than 50% of the maximum of the proposed scale. With Sweden generally being a leading nation in sustainable development the results are below of what could be expected. Ranking, based on the Swedish ranking system does not seem to predict university sustainability performance. The indication is that Gothenburg University, while having further improvement potential, could be considered a benchmark in the Swedish context.

1. Introduction

The Brundtland commission definition from 1987 states that: Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs [1]. Generally it could be said that higher education and thus universities have an essential role in advancing the understanding of requirements for sustainable development, and providing society and industries with relevant competencies in the field [2,3,4]. “An understanding of sustainability issues should be a key component of degree programmes” [5]. Those graduating today are those that will have to deal with all the challenges of sustainability. It could be expected that universities take a leading role in the promotion of sustainable development. It is therefore of interest to study how to assess university work with sustainable development. “The inherent ambiguities involved in defining sustainability and the complexities of applying the concept to diverse institutional settings have thwarted comprehensive measurement efforts until quite recently. However, cross-institutional sustainability assessment is needed to advance strong initiatives and assist lagging colleges and universities” [6]. Assessing sustainable development requires that there is an accepted understanding of what this means for universities. One of the promising approaches mentioned by [6] is the Assessment Instrument of Sustainability in Higher Education (AISHE) that defines university sustainability using the modules of identity, operations, education, research and society [7,8,9,10]. The AISHE-model provides a proposed operationalization of sustainability that clearly states responsibilities within the core activities of education and research and also in work with society, while referring to the Brundtland commission definition and the Triple Bottom Line. Like most self-assessments the AISHE-model requires a considerable organizational commitment, which could limit the number of universities using the assessment. Results from a review of the number of yearly articles on general sustainability assessment in the period 1991–2011 indicate an exponential trend in the number of papers [11]. There does not seem to be any clear convergence neither for general sustainability assessments nor for those focusing on universities. “Sustainability reporting in universities is still in its early stages (both in numbers of institutions reporting and in level of reporting) when compared to sustainability reporting in corporations” [12]. The EAUC website provides a platform for different university assessment models [13]. All the assessments mentioned in [13] are based on the university taking the lead. When this is not done it still should be possible for the interested student to make an independent assessment. Here, the access to information will be decisive and the logical place to look for this information will be the university website. An interesting question is to what extent a quick analysis of the university website could be used to provide an assessment of university activities for sustainable development.

University rankings could play a role when choosing a university. “There is no clear way to arrange campuses on a sustainability scale, yet lack of coherent criteria has not stopped campus rankings on other important issues” [6]. It could therefore be of interest to see to what extent university ranking correlates with results from a sustainability assessment.

Sweden could be seen as a country that does well in national comparisons related to sustainability. The Sustainable Society Foundation studies human, environmental and economic wellbeing on national levels and reports results in the Sustainable Society Index. In the results from 2012 Sweden ranks as nr 2 out of 151 countries studied [14]. In the Environmental Performance Index for 2012 Sweden is ranked as number 9 out of 132 countries studied [15]. OECD results from 2011 show that Sweden is nr 9 among OECD countries for GNP per capita [16]. This means that using a Triple Bottom Line approach looking at the economic, environmental and social dimensions, Sweden is well placed. The Swedish government has early identified education as a critical area for sustainable development in the university law from 1992 and which was further elaborated in 2006 [17]. Education on all levels is by law obliged to make sustainable development part of the curriculum. This indicates that there should be strong drivers for Swedish universities to work with sustainable development. Sweden is also a country with a strong culture in transparency where the principle of public access to official documents generally provides for good access of information. With a high level of computer literacy Swedish University websites generally provide a wealth of information, which possibly could be used to assess the level of university sustainability, assessed as different activities promoting behavior for achieving sustainable development.

The purpose of this paper is to propose how university sustainability could be assessed from a student perspective and based on this to make a preliminary assessment of the level of sustainability of Swedish universities. Since students could base their choices on the university ranking, the authors study if sustainability focus correlates with university ranking. Additionally we try to identify benchmark universities and superficially study the effect of university size on the proposed sustainability assessment.

2. Methods

In order to emulate the student perspective the authors have studied websites formulating a set of questions. The authors have looked at different ways of finding information on the university website. The first general approach chosen was a free search using the search term sustainable development. Here, it would have been possible to use sustainability as an alternative search term, since these often are used interchangeably. We also do that in our text. However, sustainable development is the term defined by the Brundtland commission [1]. To simplify the work we only have used sustainable development. The other general approach chosen was starting with the first university opening webpage trying to find information on sustainable development. In total three iterations with website analyses were needed to formulate the final questions, which were deemed relevant and where answers could be found using a website analysis. University sustainability could be divided in five modules as proposed in the AISHE-model [9]. Focus here is on the modules of Identity and Education. The Identity module including criteria such as transparency, policy, communication and leadership have guided the choice of questions, with questions two, three, four and partly six relating to it. Question five relates to the module Education and question six mainly to the modules of Co-operation with society, Operations and Research. The Operations module describes sustainability in managing the campus. The reason for focusing on education is that for somebody starting his or her studies, focus is believed to be on education and only later on research. In Table 1 the resulting six questions have been summarized.

Table 1. Questions used for website analysis.

Questions	Description of question to be answered by website analysis
Q1	Volume of sustainability information. Number of hits when carrying out free text search in English and Swedish on Sustainable Development—for overview only.
Q2	Relevance of general sustainability information. Average relevance of 10 first hits for Q1 (relevant if referring to education, research or university policies and not relevant when referring to historic documents or links outside of university).
Q3	Easiness to find information on sustainable development. This is based on starting from the opening webpage.
Q4	Access to and relevance of policy on sustainable development.
Q5	Number of courses offered with sustainable development in English and Swedish (Course name should include sustainable development and the course content of sustainable development should be above > 50%).
Q6	Overall management of sustainable development. Relevant mention of sustainable development in Swedish annual report (number of times, not including repetitions or headlines).

Table 1. Questions used for website analysis.

Questions	Description of question to be answered by website analysis
Q1	Volume of sustainability information. Number of hits when carrying out free text search in English and Swedish on Sustainable Development—for overview only.
Q2	Relevance of general sustainability information. Average relevance of 10 first hits for Q1 (relevant if referring to education, research or university policies and not relevant when referring to historic documents or links outside of university).
Q3	Easiness to find information on sustainable development. This is based on starting from the opening webpage.
Q4	Access to and relevance of policy on sustainable development.
Q5	Number of courses offered with sustainable development in English and Swedish (Course name should include sustainable development and the course content of sustainable development should be above > 50%).
Q6	Overall management of sustainable development. Relevant mention of sustainable development in Swedish annual report (number of times, not including repetitions or headlines).

The answer to the first question provides on overall information on the quantity of sustainability information on the website when using the website search engine. With an increasing number of hits it becomes more difficult to find relevant answers. Providing good access to core information is therefore important. Question two was formulated as the relevance of the “10 first hits” received in the search. Frequently webpages present 10 hits at a time. We have assumed that most students would stop at the first page assessed as the “10 first hits”. This means that the university interested in good communication should see that priority is given to such information that helps the interested student to assess how the university works with sustainable development. Question three looks at the easiness of finding information on sustainable development when starting with the university website. If information cannot be found easily then it will not support the interested student to choose the university. Question four looks more specifically on if a statement on sustainable development can be found and what the policy says as a sign of management commitment. Question five is for finding courses in sustainable development. Even if the number of courses could be hard to access this information is central for an interested student. Question six is answered using the annual report. This shows if sustainable development is considered on the overall managerial level and could be seen as a simple check of if the espoused policy also is enacted. Information on the size of the university, based on number of full time students, has also been collected.

In Sweden it is URANK an independent university ranking system that ranks universities [18]. The choice of universities was based on the 2009 ranking, which was the ranking available when work started. The ranking of 2011 was used to correlate with the assessed sustainability performance. Performance should here be seen as performance based on the proposed assessment model. To what extent the university performance is sustainable is another much larger topic. There were no major changes in the rankings during the years studied apart from the number four in the ranking of 2011, KTH, not being in the 2009 ranking and therefore not included in this study.

Out of 30 Swedish universities listed in the URANK 2009 the authors chose 16 universities in two groups taking the eight highest ranked and the eight lowest ranked. Two universities were later added, one as an identified benchmark and Karlstad University because of a particular interest in seeing how the university was doing. Thus the total number became 18 (a later merger of two universities reduced the total to 29). The purpose was to compare the eight highest-ranking universities with the eight lowest ranking ones. Choosing 60% of the total population of main universities listed in URANK was deemed to be sufficient for doing the preliminary test of the applicability of a website analysis and for a preliminary assessment of the level of sustainability performance in Sweden. Clustering the highest ranked and the lowest ranked and excluding the middle group was done to more clearly indicate if university ranking correlates with sustainability performance. In order not to miss any university with good sustainability work the authors identified universities that could be benchmarks for university sustainability. This was done based on proposals from persons belonging to the national university network for sustainable development, HU2, and by preliminary web searches. Universities mentioned as benchmarks for sustainable development were Gothenburg University, Mälardalen University. Gothenburg University was already on the list but Mälardalen University has been added. Blekinge Institute of Technology has earlier been ranked as number 1 in Sweden within the framework of Engineering Education and was added as the third presumed benchmark. Benchmarks are dealt with as a separate group and are not included in the other groups. This resulted in that both the high ranked and low ranked group had seven universities each. See list of chosen universities in Table 2.

Table 2. List of chosen Swedish universities listed based on ranking from 2009. Universities indicated as potential benchmarks for work with sustainable development are marked in bold.

Ranking 2009	University
1	Karolinska institutet—a medical University
2	Stockholm School of Economics
3	Swedish University of Agricultural Sciences
4	Uppsala University
5	Lund University
6	Chalmers
7	Gothenburg University (only in benchmark group)
8	Linköping University
19	Karlstad University (only included in the total average)
20	Mälardalen University (only in benchmark group)
23	University West
24	Dalarna University
25	Kalmar University (later Linnaeus University)
26	Halmstad University
27	Blekinge Institute of Technology (only in benchmark group)
28	University of Gävle
29	University of Skövde
30	Gotland University

Table 2. List of chosen Swedish universities listed based on ranking from 2009. Universities indicated as potential benchmarks for work with sustainable development are marked in bold.

Ranking 2009	University
1	Karolinska institutet—a medical University
2	Stockholm School of Economics
3	Swedish University of Agricultural Sciences
4	Uppsala University
5	Lund University
6	Chalmers
7	Gothenburg University (only in benchmark group)
8	Linköping University
19	Karlstad University (only included in the total average)
20	Mälardalen University (only in benchmark group)
23	University West
24	Dalarna University
25	Kalmar University (later Linnaeus University)
26	Halmstad University
27	Blekinge Institute of Technology (only in benchmark group)
28	University of Gävle
29	University of Skövde
30	Gotland University

The analysis of the answers for the questions in Table 1 are based on five equally weighed criteria and presented in Table 3. Each criterion is assigned a rating between 0 and 0.2. The best performance based on the model is thus 1. The proposed assessment model represents a preliminary structure to convert information from university websites to a quantified assessment of university activities for sustainable development.

Table 3. Description of interpretation of the web-analysis (SD = Sustainable Development).

Criteria	Max. rating	Resul-ting score	Comments
1a.Relevance % English for first 10 hits	10/10		Courses, research and policy are relevant but not events, links or other references—relevant when information can benefit a potential student
1b.Relevance % Swedish for first 10 hits	10/10
1.Average relevance of 1a and 1b	10/10	0.2
2.Easiness to find information	1	0.2	Max. score when SD information is found from opening page or “about the university”; SD is explained using the Triple Bottom Line (TBL) and it is clear how the university works with SD
3.Access and relevance of sustainability policy	1	0.2	Max. score when there is a specific SD-policy that relates to TBL
4.Total number of courses in SD, English and Swedish	20	0.2	Max. score when 20 courses or more are found
5.Overall management of SD	5	0.2	Max. score when sustainable development is relevantly mentioned five times or more
Total score		1

Table 3. Description of interpretation of the web-analysis (SD = Sustainable Development).

Criteria	Max. rating	Resul-ting score	Comments
1a.Relevance % English for first 10 hits	10/10		Courses, research and policy are relevant but not events, links or other references—relevant when information can benefit a potential student
1b.Relevance % Swedish for first 10 hits	10/10
1.Average relevance of 1a and 1b	10/10	0.2
2.Easiness to find information	1	0.2	Max. score when SD information is found from opening page or “about the university”; SD is explained using the Triple Bottom Line (TBL) and it is clear how the university works with SD
3.Access and relevance of sustainability policy	1	0.2	Max. score when there is a specific SD-policy that relates to TBL
4.Total number of courses in SD, English and Swedish	20	0.2	Max. score when 20 courses or more are found
5.Overall management of SD	5	0.2	Max. score when sustainable development is relevantly mentioned five times or more
Total score		1

Criterion 1—Relevance of sustainability information in a web search. This is assessed as the number of relevant entries out of the 10 first entries in the web search. Ten out of 10 scores 0.2. For relevant hits between zero and 10 the rating is assessed linearly.

Criterion 2—Easiness to find sustainability information from university website. Number of clicks needed and relevance of the information found. Zero, if no information is found and 0.2 if there is a link from the opening page or “about the university” that leads to an explanation of how university works with sustainable development that includes the Triple Bottom Line. A semi quantitative assessment of the provided information results in a rating between 0 and 0.2.

Criterion 3—Access and relevance of sustainability policy. Zero, if no policy is found and 0.2 if there is a specific and official policy on sustainable development that is based on the Triple Bottom Line. A semi quantitative assessment of the provided information results in a rating between 0 and 0.2.

Criterion 4—Total number of courses in sustainable development in English and in Swedish. Zero, if no courses are found and 0.2 for 20 courses or more. For a total between 0 to 20 the rating is set linearly.

Criterion 5—Overall management of sustainable development. Total number or relevant mentions in the yearly report. Zero, if no mention of sustainability or sustainable development and 0.2 when sustainable development is relevantly mentioned five times or more. For a total between 0 to 5 the rating is set linearly.

3. Results and Discussion

3.1. Webpage Analysis

In most cases it was possible to find the information needed, which indicates that the idea of webpage analysis could have potential as a method. The interpretation of webpage information, especially for criteria one to three was not easy and requires substantial time for finding comparable information. Web pages are structured quite differently with a varying logic and with varying user friendliness. This makes it less likely for the average student to spend the time needed. One option is for student organizations and especially those promoting sustainability to get involved in an assessment. With an initial investment in time, web sites provide a wealth of information. Currently, all sustainability assessments that the authors have found are based on the university taking the initiative (EAUC, 2013). Even if many of these models provide a thorough assessment, this is of little help if only a minority of the universities are engaged. While waiting for all universities to get their assessment working it could be worthwhile further developing the webpage sustainability assessment as a method for interested students.

The questions for the website analysis could be more clearly structured based on operational definitions of sustainable development. Criteria from the AISHE module of Identity [9] have been used by [19] to formulate questions. The Identity module defines policies and goals. There is a balance between making a good assessment of university sustainability and making it doable. The AISHE model has five modules and a total of 30 criteria. Translating all these for a website analysis is most likely not possible. The Identity module [9] could be used as a first stand-alone assessment [19]. The proposed logic by [19] is that if there is no transparency, no clear vision and policy, no clear commitment from management, no sustainability communication and no expertise in sustainable development, it is unlikely that the university works with sustainable development. This would indicate that a first review of university sustainability performance could be done based on what is espoused and how this is communicated. One possibility of improving the validity of results could be to introduce some additional interviews of contact persons, including current students.

3.2. Preliminary University Sustainability Assessment Results

The average assessed sustainability performance for the 18 studies universities is 0.46 out of a maximum of 1 defined as best performance according to the five criteria. Most universities only have a few courses in sustainable development. Many university search engines do not permit free searches of courses but require a functional topic based selection where sustainable development is not one of the areas that can be chosen. Out of the studied universities only Gothenburg and Uppsala provide students looking for a course with a declaration of which courses contain sustainable development and to what extent. Uppsala University has a continuous scale from 0 to 100% for the content of sustainable development. This could serve as a benchmark. Most universities could with little effort considerably improve the availability of information on sustainable development. The page “about the university” could include a headline for sustainable development that takes the student to the relevant information on how the university views sustainability, how they work with it and which courses can be studied. Only few universities have gone through the trouble of clearly presenting their interpretation of the Triple Bottom Line and how this affects the university education, research and societal co-operation. Here, the University of Gävle (UoG) could serve as a role model. A link to sustainable development is found on the principal webpage, which leads to the information needed. UoG is the highest performing university in the group of low ranked universities with a score of 0.69 (see Figure 1).

Figure 1. Assessed sustainability performance and university ranking.

Figure 1. Assessed sustainability performance and university ranking.

The chosen method of website analysis and the proposed scale for rating sustainability performance have not been tested before and the results are only indicative. The preliminary results need to be compared with other assessment results. Sweden should be a leading nation in working with sustainable development based on different national rankings. Additionally, universities are clearly by law obliged to work with sustainability. Sweden has some 40% of the young generation attending university education and should be a role model in educating ethically aware global citizens within fields important to sustainable development. In spite of this the average performance seems to be rather low.

If these results are representative for other leading countries within sustainable development then the global situation is worrisome. In any change, management plays a crucial role. The reason for university management not seemingly giving priority to activities proposed in the university law on sustainable development could be related to lack of active follow up and commitment from the Swedish government. Since 2006, when the law for sustainable development was issued and until mid 2013, there has been no published follow up on how universities are living up to the expectations formulated in the law.

3.3. Effects of University Ranking

Results in Table 4 indicate that there are some differences between the average of the high ranked and low ranked group of universities with a slightly higher average score for the high ranked. However, the range is wide and Figure 1 does not indicate any clear correlation between sustainability performance and ranking. Based on results in Figure 1, university ranking as defined by URANK does not seem to be a good predictor for university sustainability. The URANK-model does not include any parameters relating to sustainable development apart from some social factors relating to student recruiting, which could be one explanation to the lack of correlation. More highly ranked and prestigious universities could be expected to look after all aspects of their branding, which could turn into “sustainability washing” and in that case become a bias in the assessment. University ranking could be complemented to include whether or not universities are assessing their sustainability performance and how they are performing in this assessment [20].

Table 4. Summary of sustainability performance for chosen university groups.

Parameter	All	Low ranked	High ranked	Bench-mark
Number of universities	18	7	7	3
Total full year students 2012	10238	6766	12948	13012
Total hits on WEB page Eng—2012	1676	154	3060	2776
Total hits on WEB page Swe—2012	2136	270	3190	3332
Relevance % Eng for first 10	38	25	37	53
Relevance % Swe for first 10	51	34	56	73
Relevance average	44	29	46	63
Links to SD-information (2012)	0.4	0.3	0.5	0.5
Policy found and relevance (2012)	0.6	0.7	0.6	0.3
Courses ENG SD (2012)	3	0	4	6
Courses SWE SD (2012)	5	2	8	6
Total courses	8	2	12	11
Yearly report 2011—SD relevantly mentioned	8	6	13	6
Sustainability assessment	0.46	0.38	0.51	0.57
Range of assessment	0.00–0.77	0.10–0.69	0.00–0.75	0.41–0.77

Table 4. Summary of sustainability performance for chosen university groups.

Parameter	All	Low ranked	High ranked	Bench-mark
Number of universities	18	7	7	3
Total full year students 2012	10238	6766	12948	13012
Total hits on WEB page Eng—2012	1676	154	3060	2776
Total hits on WEB page Swe—2012	2136	270	3190	3332
Relevance % Eng for first 10	38	25	37	53
Relevance % Swe for first 10	51	34	56	73
Relevance average	44	29	46	63
Links to SD-information (2012)	0.4	0.3	0.5	0.5
Policy found and relevance (2012)	0.6	0.7	0.6	0.3
Courses ENG SD (2012)	3	0	4	6
Courses SWE SD (2012)	5	2	8	6
Total courses	8	2	12	11
Yearly report 2011—SD relevantly mentioned	8	6	13	6
Sustainability assessment	0.46	0.38	0.51	0.57
Range of assessment	0.00–0.77	0.10–0.69	0.00–0.75	0.41–0.77

3.4. Benchmark Universities

The three universities indicated as benchmarks have an average of 0.57, which is above the average performance of 0.46. This is mainly due to Gothenburg University, one of the benchmarks, having the highest score of all studied universities at 0.77. Based on the preliminary assessment only Gothenburg University could, in the Swedish context, be considered a benchmark, out of the three indicated as such. There is a possibility that some of the not studied universities could be a benchmark. Since the authors initially interviewed people within the national group HU2 working with university sustainability the possibility of the existence of an unknown Swedish benchmark university is not likely. However, there are four other universities, not earlier indicated as benchmarks that also score in the range of 0.69 to 0.75. Web pages change quite frequently and in the case of work being driven by a few enthusiasts the assessed performance could change rapidly.

3.5. Size of University

In Figure 2 the size of the university based on number of full time students is correlated with the sustainability performance. Results show a tendency for larger universities performing better in the assessment. However, there are also small universities with a good performance. Larger universities would normally have more resources making it easier to assign somebody to look after the sustainability appearance on the website, which could introduce a bias in the assessment. UoG, ranked 28 of 30 with 6,500 full time students has one of the highest ratings. This could be due to a clear management commitment manifested by the appointment of a vice rector for sustainable development.

Figure 2. Assessed sustainability performance and university size.

Figure 2. Assessed sustainability performance and university size.

4. Conclusions

The preliminary webpage assessment for sustainability information is promising. Most universities provide the information needed for making an assessment based on important criteria, such as transparency, vision and policy, expertise (courses provided) and communication, defined in, e.g., the AISHE-model for assessing university sustainability [9].

The indication is that Swedish universities based on results from 2012 are not working with sustainable development to the extent that could be expected considering the university law from 2006 that clearly defines a triple bottom line approach of sustainable development. The average assessed sustainability performance for the 18 studies universities is at about 50% of best performance according to the defined scale.

Ranking as exemplified by URANK cannot be used to predict university sustainability performance.

In the Swedish context Gothenburg University (GU) could be considered a benchmark in sustainable development. This is based on a combination of the initial information on benchmarks and the results from the preliminary assessment. Compared to general sustainability challenges, GU still needs considerable improvements.

University size does not seem to be a prerequisite for good sustainability performance. Instead one of the important prerequisites could be management commitment.

Many of the planetary problems threatening mankind and nature need to be solved by those graduating today, see for example [21]. Compared to the global requirements, relatively little seems to have happened with how universities in Sweden work with sustainable development, considering that more than 20 years have elapsed since the Rio 1992 conference and Agenda 21 and that we are at the end of the UN decade of Education for Sustainable Development (2005–2014) [22].