Investigation on Evaluation Framework of Elementary School Teaching Materials for Sustainable Development

Abstract

Education for Sustainable Development (ESD) forms part of Target 4.7 of Sustainable Development Goal 4 in the 2030 Agenda. This paper presents an effort to propose an evaluation framework of elementary school teaching materials in Taiwan for sustainable development considering three dimensions of sustainability: Environmental, social, and economic. The proposed framework comprises four levels: Lever 1, education for sustainable development; Level 2, teaching scopes; Level 3, learning indicators, and; Level 4, learning topics. This study first, through literature reviews, proposed an initial evaluation framework and then, through in-depth expert interviews, obtained a modified framework. Thereafter, the Delphi questionnaires were conducted to establish the final evaluation framework. The framework includes four teaching scopes, ten learning indicators and twenty-one learning topics. Furthermore, the weights of each scope and its associated indicators were analyzed and compared through AHP questionnaires to obtain the scoring table for sustainability teaching materials implemented in a school. Finally, the scoring table was applied to an existing elementary school to investigate its implementation of the teaching materials on sustainable development. Based on the result from the scoring table, the areas needed for improvement were identified and the improvement strategies were then proposed.

1. Introduction

Since the Industrial Revolution, the continuous development and activities of human beings on the earth have caused many changes in the environment: Global warming, ozone depletion, greenhouse effect, melting of polar icebergs, acid rain, climate anomalies, and water, air, and soil pollution. It is true that for the sake of economic development, human beings have over-exploited the existing environment, resulting in constant crises or even endangering survival. They have also realized such threats and have begun to face the problem and work hard to improve. In 1980, the World Conservation Organization first proposed “Sustainable Development (SD)” and called for global attention to the global environmental crisis [1]. The urgency of global environmental protection and sustainable development is even more evident after many various summits since the middle of the last century, including the 1972 United Nations (UN) Conference on the Human Environment in Stockholm [2]; the 1992 UN Conference on Environment and Development in Rio de Janeiro, Brazil [3]; the 2002 World Summit on Sustainable Development in Johannesburg, South Africa [4]; the 2012 UN Conference on Sustainable Development in Rio de Janeiro [5]; and the 2015 United Nations Sustainable Development Summit in New York [6].

The global interest in efforts to address sustainability challenges through education is growing steadily. According to the definition of sustainability presented by the University of Michigan, Ann Arbor in 2002 [7], sustainability, regardless of what entity is considered, involves two critical elements: (1) Ensuring that there are sufficient supplies of the ecological, material, human, and social resources necessary to allow humans to meet basic needs and to support continued development, and; (2) ensuring that access to this sufficient supply of resources is equitable both intergenerationally (among all members of the current generation) and intragenerationally (between this and future generations). Thus, sustainable development needs to meet the needs of the present without compromising the ability of future generations to meet their own needs. Accordingly, education is a key enabler of sustainable development. The role of ESD was recognized in the major UN summits on sustainable development since 1992. Sustainable development can be thought of in terms of three dimensions, i.e., the environment, the economy and the society [8,9,10,11]. Today, these three dimensions are stated in the 2030 Agenda. EDS forms part of Target 4.7 of Sustainable Development Goal 4, which by 2030, aims to ensure that all learners acquire the knowledge and skills needed to meet the needs using a balanced and integrated approach to the economic, social and environmental dimensions of sustainable development and is understood as an important means to achieve all the other Sustainable Development Goals [12]. However, all the SDGs contribute to the achievement of Goal 4.

In order to comply with the global trend, Taiwan established the Commission on Sustainable Development in 1996 to come up with policy guidelines and programs. The concept of green building and its corresponding strategies were thus initiated by the Council for Economic Planning and Development and have been incorporated into the national development plan as one of the most effective measures. The Construction and Planning Agency of the Ministry of the Interior officially declared to promote the green building policy in the White Paper of Construction, as well as in the White Paper of Environment of the Environmental Protection Agency. Also, in 1998, the Architecture and Building Research Institute, Ministry of the Interior, launched the “green building and living environment technology plan” and has been continually implementing relevant programs [13]. In May 2002, Taiwan passed the “Challenge 2008: Six-year National Development Plan”. The green building and green campus programs are two of its many projects. In the green campus project, the Ministry of Education (MOE) proposed a “Taiwan Sustainable Campus Project”, which includes two components, the “Sustainable Campus Program” and the “Green School Partnership Program”. The Sustainable Campus Program, a “hardware” reform, has called for proposals, from each school, to renovate the campus from the following aspects, such as energy-saving appliance, water recycling and reuse system, permeable ground surface, and artificial wetland, multi-layer green for CO₂ reduction and biodiversity, compost from foliage and kitchen waste, educational organic farm or eco-pond, among others. The Green School Partnership Program, a “software” reform, is a system designed to assist the development of “Green Schools” in Taiwan providing concepts, action plans, instructional materials, and government and private resources for developing Green Schools [14]. Eventually, the two programs become a part of ESD in Taiwan, which also include the other three approaches for conducting programs of EDS: Developing ESD instructional materials, holding international conference of SD education, and conducting teacher workshops for SD education. The three main objectives of ESD in Taiwan includes (1) introducing and incorporating the concept of SD into the school education and citizen’s daily life; (2) coordinating the resources of governmental agencies, private sectors, business, and schools to implement education for sustainable development, and; (3) international cooperation of education for sustainable development [15].

In the face of today’s environmental problems, we need to think about how to face and how to reduce the recurrence of the environmental crisis and its corresponding solutions. Many scholars have started professionally promoting environmental protection plans and practices to prevent problems. In order to find out how to solve the problem, in addition to considering the actual problem, on the other hand, it should be based on “prevention of the problem”. Therefore, internalizing the concept of coexistence of sustainable environment and ecological balance to people’s hearts can reduce the occurrence of future problems. The important promoters of this must rely on the power of “education”. Starting from the children to promote related concepts to their life, they can feel the environment from an early age, can care and pay attention to the surrounding environment. Here we are going to focus on the children in the elementary school.

Määttä et al. [16] proposed five cornerstones of Sustainability Education (SE) concerning learning and teaching of SE in schools and organizations: why, what, how, who and when. Some of the answers to “Why do we need SE?”, “What does SE consist of?”, “How to implement SE?” can be found in the 2030 Agenda. However, there are no simple answers to “What does SE consist of?” and “How to implement SE?” since SD issues are complex due to the interrelations among the three pillars [17] and the concept of SD can be understood in various ways, according to different views [18]. A lot of different studies on “how to implement SE?” from pre-school to higher education have been reported in the literature [19,20,21,22,23,24,25,26,27,28,29,30]. These studies consist of the following activities to promote learning, including the use of the school garden or ecological garden [19,20,21], the use of storytelling [22], measuring student sustainability competencies [23], using systems thinking through a participatory approach [24], the use of active learning activities for recycling [25], the use of educational games [26], extracurricular activities of reflective learning [27], flipped classroom as an active learning methodology [28], designing powerful learning environments [29], or designing student’s action competence [30]. As to “What does SE consists of?”, there are some studies [31,32,33,34,35]. For examples, Martínez-Medina and Arrebola [32] had studied the sustainability activities in Spanish elementary education textbooks. Also, Fredriksson et al. [33] had performed a comparative study of curriculums for education for sustainable development in upper secondary schools in Sweden and Japan. However, they did not propose comprehensive teaching materials or evaluation framework suitable for students in their academic system. Here lies the reason for this work.

In addition to a series of earlier external environmental hardware promotion, in June 2008, the Department of Primary Education, MOE, announced the “2008 Grade 1–9 Curriculum Guidelines” [36]. One of the key issues is “Environmental Education”. In November 2014, the K-12 Education Administration, MOE, announced the “Curriculum Guidelines of 12-Year Basic Education—General Guidelines”, in April 2018, the “Curriculum Guidelines of 12-Year Basic Education—Life Subject”, and in November 2018, the “Curriculum Guidelines of 12-Year Basic Education—Natural Sciences Subject” [37]. One of the key issues in the Life subject is “Environmental Education” and in the Natural Sciences subject is ‘Sustainable Development in Nature”. The SD teaching materials of the two subjects in the textbooks will be examined.

The green building certification system of Taiwan has been in operation since 1999. EEWH (Ecology, Energy Saving, Waste Reduction and Health) represents the four major issues in an ideal green building [38,39]. Although the connotation of green building is mainly from the perspective of the building, these four issues are still valid for the study of sustainable development because EEWH has considers not only the building itself but also its interior and exterior environment, and the execution of the four issues more or less involves the economic and social dimensions. Yu et al. [40] had studied the three dimensions of SD and their related indicators by examining eight EEWH certified green buildings and three infrastructure projects using ecological construction methods. For elementary school students, the social and economic dimensions could be considered from the impact of ESD on their daily life. Therefore, the sustainable development should be considered from the perspectives of the building, the environment and the daily life. However, since the current teaching materials mostly implement the concept of sustainable development based on the Green Building Evaluation Manual [38], it is important to include the teaching materials having the SD impact on the daily life. This study chooses elementary school education as the starting point, uses the issues and indices in the Green Building Evaluation Manual as the reference direction, cooperates with the MOE education curriculum guidelines, and reviews the textbooks of the two aforementioned subjects to establish an evaluation framework of SD teaching materials suitable for the elementary school students in order to provide them with the learning direction of the concept of sustainable development, hoping that the concept of “sustainability” is internalized in the hands of education, the environment is treated with a positive attitude, and good habits of respect for life are established.

2. Methodology and Procedure

This paper presents an effort to propose an evaluation framework of elementary school teaching materials for sustainable development considering three dimensions of sustainability: Environmental, social, and economic. Using the three-level (Level 1, green building goal; Level 2, green building issues; and Level 3, green building indicators) EEWH evaluation framework as a basis, the proposed framework comprises four levels: Lever 1, education for sustainable development; Level 2, teaching scopes; Level 3, learning indicators; and Level 4, learning topics. The research procedure adopted in this study is described below. It includes the following steps and methods:

(1)

Identifying candidate teaching scopes, learning indicators and learning topics: Candidates to be used in the formulation of an initial evaluation framework were identified through reviews of scientific references, education curriculum guidelines, regulations, and current textbooks with sustainability contents;

(2)

Pre-screening candidates through expert interviews: semi-structured in-depth expert interviews were conducted until a consensus was reached in order to obtain a modified proposed evaluation framework;

(3)

Establishing the final evaluation framework by using the Delphi method: A few rounds of expert questionnaires based on the modified proposed evaluation framework were conducted until a consensus was reached to obtain the final evaluation framework;

(4)

Determining the importance of each scope and indicator by using the Analytic Hierarchical Process (AHP): The weights of each scope and its associated indicators were analyzed and compared through AHP questionnaires to obtain a scoring table for sustainability teaching materials implemented in a school;

(5)

Applying the evaluation framework to an exemplary school: the scoring table was applied to an existing elementary school campus to examine how the teaching materials for sustainable development were implemented.

3. Development of Evaluation Framework

3.1. Identifying Candidate Teaching Scopes, Learning Indicators and Learning Topics

The candidate teaching scopes, learning indicators and learning topics to be used in the initial evaluation framework were identified mainly through the following literature reviews: (1) Green Building Evaluation Manual [38], (2) MOE education curriculum guidelines [36,37], (3) The status of education for sustainable development in Taiwan [15], (4) Manual for sustainable campus planning, design and management [41], (5) Sustainable campus construction guide [42], and; (6) the high current usage elementary school’s textbooks with contents that are related to sustainability.

This study uses the evaluation system in the Green Building Evaluation Manual as a basis to develop the evaluation framework. The teaching scopes (Lever 2) use the four issues in Ecology, Energy Conservation, Waste Reduction, and Health (EEWH). (Figure 1)

The learning indicators (Level 3) use the nine indices in EEWH, Biology, Vegetation Amount, Permeable Lot, Energy Conservation, CO₂ Reduction, Waste Reduction, Indoor Environment, Water Resources, and Sewage and Garbage Improvement. However, the Energy Conservation index is further divided into the Energy Usage and the Energy Saving learning indicators. For ESD, the meaning of each learning indicator is somewhat different from what is in the Green Building Design Manual. The meaning of each learning indicator is explained below.

Biodiversity: The original meaning is to protect the biological living environment at the bottom of the “ecological pyramid”. The teaching perspective is to cultivate a positive attitude towards the environment through understanding the environment and basic concepts of ecology.

Vegetation amount: The original meaning is to use the natural soil layer in the building lot and the covering layer on the roof, balcony, and exterior wall to plant various plants. The teaching perspective is to understand the basic concepts of the environment and plants, and the plants’ contribution to the environment, and to grow more plants that are good for the environment.

Permeable lot: The original meaning refers to the ability of the natural soil layer and artificial soil layer in the building lot to conserve water and to retain rainwater; the better the water retention performance of the lot, the better the ability of the lot to hold rainwater, which is beneficial to the activities of soil microorganisms, thereby improving soil activity to maintain the natural ecological environment balance in the building lot. The teaching perspective is to understand the phenomenon of water movement, understand the difference between soil and general ground water, and understand the concept that water directly penetrates into the soil ground to achieve the most natural and environmentally friendly cycle.

Energy usage: The original meaning is based on the air-conditioning and lighting power consumption as the main assessment objects. However, the learning indicator focuses not only in the energy usage in a building but also in our daily life. The teaching perspective is to understand energy, its source and application, its importance in daily life and its impact on human beings.

Energy saving: The original meaning is to focus on the energy saving in the life cycle of 50 to 60 years of a building, which consumes a lot of energy from all stages of building material production, construction and transportation, daily use, maintenance, and demolition. The daily energy consumption of air conditioning, lighting, elevators, etc. for long-term use accounts for the largest part, and in the daily energy consumption, air conditioners and lighting in a building account for the largest proportion of electricity consumption. In summer, the ratio of electricity consumption for air conditioning in buildings is about 40 to 50%, and the electricity ratio of lighting is as high as 30 to 40%. It is without doubt that the building energy saving from air conditioning and lighting is the most effective. On the other hand, due to the long service life of the building, its cumulative energy saving effect is far better than other industrial products. However, the learning indicator focuses not only in the energy saving in a building but also in our daily life. The teaching perspective is to understand the energy saving and environmental issues and their impact on human beings, and then form the habit of environmental protection actions and perform environmental protection activities of individuals or groups.

CO₂ reduction: The original meaning is the CO₂ emissions converted from the energy used in the production process of all building materials. However, for the elementary school students, the learning indicator focuses not in the CO₂ reduction on the building materials but rather in our daily life. The teaching perspective is to understand the impact of CO₂ on the environment, the current situation of the problems and its impact on humans, let school children exercise the ability to reduce CO₂ in daily life environmental action.

Waste reduction: The original meaning is the reduction of the waste, including the engineering imbalance earthwork, spoils, abandoned building materials, scattered dust, etc. generated by the construction and subsequent demolition process, which are enough to damage the surrounding environment and human health. However, for the elementary school students, the learning indicator focus not in the waste reduction in building construction but rather on the waste reduction in our daily life. The teaching perspective is to observe the waste generated in our daily life, and let students think about ways to reduce the generation of waste.

Indoor environment: The original meaning is to evaluate the environmental factors that affect the health and comfort of living in the indoor environment, such as soundproof, lighting, ventilation, indoor decoration, indoor air quality, etc. The learning indicator focuses on the two parts of lighting and ventilation. Others, such as decoration and soundproof, that belong to the building hardware and equipment are not considered in the elementary school stage education. The teaching perspective is to understand the existing environmental issues through understanding of basic concepts such as light and wind, the current situation and its impact on humans, and in turn increase the sensitivity to light and ventilation in daily life.

Water resources: The original meaning is to focus on the ratio of the actual water consumption to the average water consumption of the building. Its water consumption assessment includes water efficiency assessment of kitchens, bathrooms, and water taps, and rainwater and reclaimed water reuse. The teaching perspective is to understand the basic concepts of water, the interaction between water, ecology and the environment, and the status of water and environment problems and its impact on human beings so that students can develop the habit of environmental protection in daily life, perform individual or group environmental protection activities, and be friendly to the environment.

Sewage and garbage improvement: The original meaning is to focus on specific evaluation items related to building space facilities and use management. It is an evaluation indicator that allows owners and users to specifically control and improve environmental hygiene. The teaching perspective is to understand the basic concepts of the environment, understand the status of environmental problems and its impact on human society and culture so that students can develop the habit of garbage classification in daily life, perform individual or group environmental protection activities, and be friendly to the environment.

The sustainability teaching materials can be found in the 1st and 2nd grades’ Life Subject textbooks and the 3rd to 6th grades’ Nature and Life Technology Subject textbooks. The high current usage elementary school’s textbooks published by Kangxuan, Hanlin, and Nanyi publishers [43,44,45] were considered in this study to develop the learning topics associated with their learning indicators. A total of 36 textbooks were examined. The sustainability connotation of units in a textbook associated with each of the learning indicators, whether it is poor, fair or acceptable, were determined. First, the authors examine the teaching material of a learning indicator is included in the textbook, and then how much. As to the covering depth, since the textbooks were approved by the government, they should have the right depth. After discussing the obtained examining results among the authors, if there was any doubt, the life and nature science teachers in the elementary school were consulted in order that

Table 1. Different degree of coverage in the textbooks related to the learning indicators.

Publisher	Grade\Semester Textbook	Biodiversity	Vegetation Amount	Permeable Lot	Energy Usage	Energy Saving	CO2 Reduction	Garbage Reduction	Indoor Environment	Water Resources	Resource Usage
Kangxuan	1st /Fall								○
	1st /Spring		○							○
	2nd/Fall								○	○
	2nd/Spring	○								○
	3rd/Fall		◎						◎
	3rd/Spring	◎	◎							◎
	4th/Fall	◎	○		◎	◎			○
	4th/Spring	◎			◎
	5th/Fall		●						◎	◎
	5th/Spring	◎					◎		◎		○
	6th/Fall					○			○	○
	6th/Spring	●
Hanlin	1st /Fall		○						○
	1st /Spring									○
	2nd/Fall	○							○
	2nd/Spring	○	○					◎		○
	3rd/Fall		◎						◎
	3rd/Spring	◎	◎							◎
	4th/Fall	◎			◎	◎				○
	4th/Spring	◎							○	○
	5th/Fall		◎			○	○		◎
	5th/Spring	◎								○
	6th/Fall								○	○	○
	6th/Spring	●	●		◎	◎	◎	◎		◎	◎
Nanyi	1st /Fall								○
	1st /Spring									○
	2nd/Fall								○
	2nd/Spring	○	○							○
	3rd/Fall		◎						◎
	3rd/Spring	◎	◎							◎
	4th/Fall	◎			●	◎			○
	4th/Spring	◎								○
	5th/Fall		◎		◎	◎	○		◎
	5th/Spring	◎
	6th/Fall		◎			◎			◎	◎
	6th/Spring	◎	◎		◎	◎	◎	◎	◎	◎	◎

Note: Legend: ●—Good; ◎—Fair; ○—Poor.

can be obtained. It can be seen that not all the sustainability contents to be given in the learning indicators are covered, especially in the permeable lot, garbage reduction and resource usage indicators. Also, many of the sustainability contents in the textbooks are fair or poor with only a few contents having an acceptable coverage of sustainability content. It is without doubt that the sustainability teaching scope had not been comprehensively considered when the textbooks were edited.

The contents in these textbooks served as a starting basis to organize the learning topics associated with their related learning indicators. However, there are different educational resources to learn and teach about sustainability, especially in those indicators not covered or not covered enough in the textbooks. In-class or extracurricular activities can be designed to develop students’ action competence for a sustainable future. Reviewed studies by Chen and Liu [30] indicate that action-oriented pedagogy cultivate students to be active participants, empower their capability of deliberating the causes and effects, and construct their vision for finding strategies toward the problems. The teaching topics of action competence can be a recycling activity [25], an outdoor teaching in an ecological garden [19], or A do it yourself (DIY) project using recycling materials. They can also relate to the current domestic and international environmental issues, or the promoting policies and campaigns to save the planet and save energy. All can be incorporated into the teaching materials. For examples, “World Car Free Day”, which is celebrated on 22 September, encourages motorists to give up their cars for a day and calls on everyone to take public transportation, ride bicycles or even take a walk, hoping to awaken the public’s attention to environmental protection [46]; “Earth Hour” is a worldwide movement organized by the World Wide Fund for Nature. The event is held annually encouraging individuals, communities, and businesses to turn off non-essential electric lights, for one hour, from 8:30 to 9:30 p.m. on a specific day towards the end of March, as a symbol of commitment to the planet [47].

The teaching materials considered in the learning topics include the following: (1) The teaching materials in the textbooks adopted by a school, (2) well-documented stainability lecture notes by teachers, (3) in-class or extra-curricular activities, (4) DIY projects or assignments, and (5) field trips. The 29 learning topics associated with their associated learning indicators were proposed. The proposed initial evaluation framework is shown in

Table 2. Initial four-level hierarchy evaluation framework.

Goal (Level 1)	Teaching Scopes (Level 2)	Learning Indicators (Level 3)	Learning Topics (Level 4)
Education for Sustainable Development	Ecology	Biodiversity	Knowing the nature
			A look at the animal world
			Biology and the environment
			Endangered species
		Vegetation amount	A look at the plant world
			Plants and the environment
			Gardens and parks
		Permeable lot	Close to the earth
			Wonderful phenomenon of water
			Ecological pond
	Energy Conservation	Energy usage	Where is the energy
			Application of energy
		Energy saving	World blackout day
			Convenient ride sharing
			Mass transportation
	Waste Reduction	CO2 reduction	Vegetable day
			Cycling day
		Waste reduction	Where does the waste come from
			I have reduced my trash
	Health	Indoor environment	Sun and life
			Light my house
			Here comes the wind
		Water resources	Water in life
			Water saving campaign
			Rainwater reuse
		Sewage and garbage improvement	Garbage classification
			Useful garbage
			Resource recycling
			Flea market

.

3.2. In-Depth Expert Interviews

After the teaching scopes, learning indicators and learning topics in the initial evaluation framework were identified, semi-structured in-depth expert interviews were conducted until a consensus was reached in order to obtain a modified proposed evaluation framework. This step can facilitate the convergence of the next step Delphi questionnaires. The interviewees were divided into teachers with more than 10 years of teaching experience in Life and Nature subjects, and experts with background in environmental education and green buildings. A total of 20 interviewees were interviewed. The interviews were mainly in person interviews. After the interviews, the suggestions of experts and scholars were collected. They suggested that the teaching scopes be unchanged; the “Sewage and garbage improvement” indicator change to “Resource usage”; in the proposed learning topics, “Knowing the nature” change to “The mystery of nature”, “Sun and life” changes to “Light and life”, “Here comes the wind” changes to “Playing games with the wind”, “Water in life” change to “Wonderful water”, and “Endangered species”, “Gardens and parks”, “Ecological pond”, “Mass transportation”, “Light my house”, “Rainwater reuse” and “Useful garbage” be deleted since their teaching materials can be presented in the other topics. The proposed learning topics were modified from 29 to 22. After the in-depth interviews, the modified evaluation framework is shown in

Table 3. Modified four-level hierarchy evaluation framework.

Goal (Level 1)	Teaching Scopes (Level 2)	Learning Indicators (Level 3)	Learning Topics (Level 4)
Education for Sustainable Development	Ecology	Biodiversity	The mystery of nature
			A look at the animal world
			Biology and the environment
		Vegetation amount	A look at the plant world
			Plants and the environment
		Permeable lot	Close to the earth
			Wonderful phenomenon of water
	Energy Conservation	Energy usage	Where is the energy
			Application of energy
		Energy saving	World blackout day
			Convenient ride sharing
	Waste Reduction	CO2 reduction	Vegetable day
			Cycling day
		Waste reduction	Where does the waste come from
			I have reduced my trash
	Health	Indoor environment	Light and life
			Playing games with the wind
		Water resources	Wonderful water
			Water saving campaign
		Resource usage	Garbage classification
			Resource recycling
			Flea market

.

3.3. Delphi Questionnaires

In order to make the evaluation framework after in-depth interviews more complete, the Delphi method questionnaire was used to analyze and review the content integrity of the framework. A panel of 20 anonymous scholars and experts was invited to provide guidance and modifications. When performing the questionnaires, those items in the framework still could be added, deleted or modified if necessary. After three rounds of Delphi questionnaires, the 22 learning topics had been modified to 21. “Biology and the environment” learning topic was renamed to “Animal and the environment”. In addition, the “Flea market” learning topic was deleted because the experts suggested its concept can be summarized under “Waste reduction”. The four-level (A, B, C, Learning topics) hierarchy of the final evaluation framework is shown in

Table 4. Final four-level hierarchy evaluation framework.

Goal (Level 1)	Teaching Scopes (Level 2)	Learning Indicators (Level 3)	Learning Topics (Level 4)
A: Education for Sustainable Development	B1: Ecology	C1: Biodiversity	The mystery of nature
			A look at the animal world
			Animals and the environment
		C2: Vegetation amount	A look at the plant world
			Plants and the environment
		C3: Permeable lot	Close to the earth
			Wonderful phenomenon of water
	B2: Energy conservation	C4: Energy usage	Where is the energy
			Application of energy
		C5: Energy saving	World blackout day
			Convenient ride sharing
	B3: Waste Reduction	C6: CO2 reduction	Vegetable day
			Cycling day
		C7: Waste reduction	Where does the waste come from
			I have reduced my trash
	B4: Health	C8: Indoor environment	Light and life
			Playing games with the wind
		C9: Water resources	Wonderful water
			Water saving campaign
		C10: Resource usage	Garbage classification
			Resource recycling

.

3.4. Sustainability Dimensions

The relationship between the teaching materials to be developed in the learning topics and the three sustainability dimensions needs to be considered when developing the framework. Environmental sustainability means that we need to ensure that we are consuming our natural resources at a sustainable rate. Economic sustainability requires that a school limits the waste of resources so that it can operate in a sustainable manner to reduce an operational cost. Social sustainability is much more difficult to define from the ESD perspective. According to the study in the University of Michigan, Ann Arbor [7], the social dimension includes the following principles: Trust, reciprocity norms, equity, and other conditions that permit coordination and cooperation for mutual benefit. This dimension can be achieved through the involvement and improvement of school children. Through the ESD activities, children in the school accept the responsibility of consistent growth and improvement. Diverse activities should be promoted and encouraged within the school. Also, social cohesions—processes, systems, and structures are provided to promote connectedness and inclusion within and outside the school. A series of extracurricular activities can be designed to satisfy this dimension. Since the three dimensions of sustainability are intertwined with each other, embedding the social dimension within a school will help support environmental and economic dimensions. For example, the extracurricular activity of monitoring the water bill with students monthly and make this activity a part of water conservation education and reducing a school’s operation cost. The suggested sustainability dimensions for the teaching materials in the learning topics are shown in

Table 5. Suggested sustainability dimensions for the teaching materials in learning topic.

Learning Indicators (Level 3)	Learning Topics (Level 4)	Sustainability Dimensions
C1: Biodiversity	The mystery of nature	Environmental
	A look at the animal world	Environmental
	Animals and the environment	Environmental
C2: Vegetation amount	A look at the plant world	Environmental
	Plants and the environment	Environmental
C3: Permeable lot	Close to the earth	Environmental
	Wonderful phenomenon of water	Environmental
C4: Energy usage	Where is the energy	Environmental
	Application of energy	Environmental
C5: Energy saving	World blackout day	Environmental, Social, Economic
	Convenient ride sharing	Environmental, Social, Economic
C6: CO2 reduction	Vegetable day	Environmental, Social, Economic
	Cycling day	Environmental, Social, Economic
C7: Waste reduction	Where does the waste come from	Environmental
	I have reduced my trash	Environmental, Social
C8: Indoor environment	Light and life	Environmental
	Playing games with the wind	Environmental
C9: Water resources	Wonderful water	Environmental
	Water saving campaign	Environmental, Social, Economic
C10: Resource usage	Garbage classification	Environmental, Social, Economic
	Resource recycling	Environmental, Social, Economic

.

3.5. AHP Questionnaires

After having establishing the elements of each level in the evaluation framework, it is important to have an idea of the distribution of its teaching materials. The AHP questionnaires, conducting with the same panel above, were used to identify the relative weights of teaching scopes of level 2 and learning indicators of Level 3. Two types of pair-wise comparisons matrices had been established: (1) The evaluation matrix of level 2 teaching scopes with respect to the goal (A), and; (2) the evaluation matrix of the learning indicators with respect to their related teaching scope at level 2. The evaluation matrix of scopes to the goal is shown in

Table 6. Evaluation matrix of level 2 scopes with respect to A.

A	B1	B2	B3	B4	Geometric Mean	Weighted Value
B1	1.000	2.453	2.051	1.052	1.517	0.367
B2	0.408	1.000	0.973	1.563	0.887	0.215
B3	0.488	1.028	1.000	1.472	0.927	0.224
B4	0.951	0.640	0.679	1.000	0.802	0.194
λmax = 4.179; C.I. = 0.060(≒0, ok); C.R. = 0.066(<0.1, ok)

. At level 3, four evaluation matrices were required to perform. The evaluation matrices of the learning indicators with respect to the related teaching scope are shown respectively from

Table 7. Evaluation matrix of indicators with respect to B1.

B1	C1	C2	C3	Geometric Mean	Weighted Value
C1	1.000	2.600	3.050	1.994	0.582
C2	0.385	1.000	1.500	0.832	0.243
C3	0.328	0.667	1.000	0.602	0.176
λmax = 3.007; C.I. = 0.003(≒0, ok); C.R. = 0.006(<0.1, ok)

,

Table 8. Evaluation matrix of indicators with respect to B2.

B2	C4	C5	Geometric Mean	Weighted Value
C4	1.000	0.333	0.577	0.250
C5	3.000	1.000	1.732	0.750
λmax = 2.00; C.I. = 0.000(≒0, ok); C.R. = 0.000(<0.1, ok)

,

Table 9. Evaluation matrix of indicators with respect to B3.

B3	C6	C7	Geometric Mean	Weighted Value
C6	1.000	0.378	0.615	0.274
C7	2.644	1.000	1.626	0.726
λmax = 2.00; C.I. = 0.000(≒0, ok); C.R. = 0.000(<0.1, ok)

and

Table 10. Evaluation matrix of indicators with respect to B4.

B4	C8	C9	C10	Geometric Mean	Weighted Value
C8	1.000	0.427	0.491	0.594	0.186
C9	2,342	1.000	1.233	1.424	0.445
C10	2.037	0.811	1.000	1.182	0.369
λmax = 3.001; C.I. = 0.000(≒0, ok); C.R. = 0.000(<0.1, ok)

. Based on the relative weights for the 4 scopes at Level 2 and the 10 indicators at Level 3, the priority weights and rankings of indicators can be established (see

Table 11. Learning indicator priority weights and rankings.

Goal (Level 1)	Teaching Scopes (Level 2)	Learning Indicators (Level 3)	Priority Weight	Rank
A:Education for Sustainable Development	B1: Ecology (36.7%)	C1: Biodiversity	21.4%	1
		C2: Vegetation amount	8.9%	4
		C3: Permeable lot	6.4%	7
	B2: Energy Conservation (21.5%)	C4: Energy usage	5.4%	9
		C5: Energy saving	16.1%	3
	B3: Waste Reduction (22.4%)	C6: CO2 reduction	6.1%	8
		C7: Waste reduction	16.3%	2
	B4: Health (19.4%)	C8: Indoor environment	3.6%	10
		C9: Water resources	8.6%	5
		C10: Resource usage	7.2%	6

). The priority weights in the table indicate that the more weight in a teaching scope or a learning indicator is the more teaching materials should be given. However, the relative weights of learning topics in Level 4 were not identified here. If they are identified, it will make the learning topic much more rigid than flexible for teachers to develop teaching materials. It is noted here that no matter how much the priority weight of a learning indicator is, the learning topics in each indicator should be implemented in order that a comprehensive teaching materials can be given to students.

3.6. Discussion

From Table 11, among the four teaching scopes, “Ecology” is the most important scope with a weight of 36.7%; followed by “Energy Conservation”, “Waste Reduction” and “Health”. It indicates that more teaching materials of sustainability education should be covered in “Ecology”, followed by “Energy Conservation”, “Waste Reduction” and “Health”. Table 11 also shows the weighting rank of each learning indicator, indicating the proportion of teaching materials or the current status of school teaching implementation in sustainability education. Among the 10 teaching indicators, “Biodiversity” is the most important indicator with a weight of 21.4%. Its teaching weight is the highest, followed by the teaching indicators of “Waste reduction” and “Energy saving”, second and third in teaching weight. Through the usual advocacy and promotion, these two indicators can become a good habit for students; the lowest weighting rank is “indoor environment”. This learning indicator, which focuses more on the planning and improvement of hardware equipment, has the lowest in teaching weight.

This study also proposes the following suggestions: (1) The framework in this study may be adjusted in teaching due to differences in school attributes, regionality, and teachers’ selection of teaching materials, as well as regional cultures and student life experience differences. In terms of breadth and depth, the design of this framework provides the direction of elementary school teaching materials for sustainable development. (2) The learning indicators and topics may be different due to the characteristics of different regions and the perspectives of the professional fields of experts. The scope of the teaching indicators and topics can be broadened or made more detailed classification; it can also be extended according to the current situation of different environmental problems; later it can also respond to the national elementary school learning stages, respectively, low, middle and high grade, or the first to sixth grade stage, and then explore more detailed staged learning topics and contents.

4. A Case Study

In order to easily apply the evaluation framework to a school, a sustainability scoring table (see

Table 12. Sustainability scoring table for the study case.

Goal (Level 1)	Teaching Scopes (Level 2)	Learning Indicators (Level 3)	Altered Weight	Given Score	Score
A: Education for Sustainable Development	B1: Ecology	C1: Biodiversity	2.14	6.9	14.77
		C2: Vegetation amount	0.89	7.4	6.59
		C3: Permeable lot	0.64	5.1	3.26
	B2: Energy Conservation	C4: Energy usage	0.54	8.7	4.70
		C5: Energy saving	1.61	7.2	11.59
	B3: Waste Reduction	C6: CO2 reduction	0.61	6.3	3.84
		C7: Waste reduction	1.63	8.9	14.51
	B4: Health	C8: Indoor environment	0.36	8.1	2.92
		C9: Water resources	0.86	6.9	5.93
		C10: Resource usage	0.72	7.4	5.33
Total Score					73.44

Notes: 1. How to fill in the form: Give each indicator a score from 0–10 integer number. Score 0–4: Not acceptable and needs to be improved immediately. Score 4–7: Partially acceptable but can be improved. Score 7–10: Acceptable. 2. Indicator score = Altered weight × Given score. 3. Total score is an indicator of the sustainability of a school campus. 4. Improvement criteria: No or minimum Improvement: Total score is between 80 and 100. Some Improvement: Total score is between 60 and 80. Immediate Improvement: Total score is less than 60.

) was established based on Table 11. For the sake of letting a user give the score to a learning indicator easier, the absolute weight in the table is changed to an altered weight. Depending on how the implementation and performance of teaching materials in an indicator is, the user can give an indicator a score from 0 to 10. The score for each indicator is equal to its altered weight times its given score. An exemplary study case in Taichung city, Taiwan, is presented to show how it was used to do sustainability teaching materials evaluation.

School Profile: The exemplary School is located in central Taiwan, and is located approximately in the northern part of the Taichung Basin. The suburban area of the area is close to farmland and ordinary homes. It is a learning place with simple folk customs and beautiful environment. The total campus area is about 23,000 square meters, and the total campus floor area is about 12,000 square meters. It currently has 34 classes, including special education and kindergarten classes. It is a medium-sized school with more than 900 students. There are more than 70 ordinary and specialized classrooms. The school has a well-equipped gymnasium, basketball court, entertaining game facilities and a green campus.

4.1. SD Teaching Overview

The teaching overview associated with sustainable development is briefly given below.

Ecology Scope: Cooperating with the content of the textbooks and the school environment, students start to understand and explore nature from the campus. Figure 2 shows that the students tour the campus to explore and feel its natural ecology.

Energy Conservation Scope: The school moves towards energy conservation from improving hardware equipment and promoting the energy saving concepts from everyday life, such as the library and computer classrooms in the building are designed so that light can penetrate into the classroom to achieve brightness and energy saving; if the old light fixture in the room is unusable, it is replaced with a T5 fixture during repair, which saves electricity compared to the original one; install the LED lights wherever it is possible; the windows are opened to ventilate when arriving at school in the morning; the lights are turned on in time according to the brightness of the classroom, turned off when cleaning, dining or leaving the classroom.

Waste Reduction Scope: The school has used the following methods or activities to achieve the goal of waste reduction, including the Monday’s Vegetable Day policy is promoted to reduce CO₂; ordinary time is used to propagate the categories of daily necessities; resource recycling classification information is posted in the bulletin board; each class is equipped with resource recycling bins and trash cans; recycling bins are divided into containers (paper containers, plastic containers), paper and plastic for recycling; resources recycling work is executed at a fixed time every week; also to encourage recycling, waste batteries can be collected at the office to exchange for rewards; in addition, because there are a large number of fallen leaves and branches in the campus, a fixed collection location is set up to collect them in the campus and then, the district office collects and transports them to the mountain nearby for composting. Figure 3 shows the waste reduction methods used and an advocacy activity of waste batteries recycling.

Health Scope: The textbook knowledge and daily life experience are combined for students to learn in this scope, such as: the windows are opened to ventilate when arriving at school in the morning; the lights are turned on in time according to the brightness of the classroom; some of the faucets in the handwashing sink are replaced with water-saving faucets to achieve the purpose of saving water from the hardware equipment; an empty bucket is put on the handwashing stand outside the middle and upper grade toilets to collect water for washing hands or things so that it can be used for cleaning the toilets in the afternoon; an empty bucket is put on the handwashing stand outside the lower grade toilets (in the first floor) to collect the water after washing the dust cloth so that it can be used to water onto the turf outside the flower garden; in addition, cooperating with learning content, empty bottles recovered from resources can be made into environmentally-friendly musical instruments or toys. Figure 4 shows resource saving or reuse.

4.2. Sustainability Score for the Study Case

Five teachers in the school with at least 10 years teaching experience in the Life and Nature subjects were invited to score the sustainability scoring table. For the score of a learning indicator given below 7, an expert was asked to write down his or her opinions and suggestions for future improvement. The score for each indicator is the average of the scores given by them. The learning indicator scores and total score for the exemplary elementary school are shown in Table 12. There are four indicators to be improved. The total score is 73.44, indicating some sustainability teaching improvement is needed.

4.3. Improvement Strategies

The improvement strategies suggested are described below.

Biodiversity: A lot of common living creatures cannot be observed in the existing school environment, supplementary teaching materials such as videos and books should be adopted and students are encouraged to go outdoors and get in touch with nature.

Permeable lot: There is a lack of teaching materials in this indicator. The pavement in the school is a good place to teach students the basic knowledge on permeable materials. Chain bricks used in the campus have limited water permeability, but they are flatter than grass-grown hollow bricks. However, they were used to take care of the safety of school children’s activities. When rebuilding the new building, careful consideration may be given to finding a balance between the safety of the children and the site water retention.

CO₂ reduction: Due to school children’s limited ability. CO₂ reduction is a good way to start from the vegetable day, but it is more important for children to understand the reasons behind it and think about other ways to reduce carbon in life. The teaching materials for common reusable or recyclable materials should be increased to let students be aware of them.

Water resources: Since lower grades often forget to turn off the faucet and waste water, the one-day activity of using very limited water can let them understand the suffering of waterlessness and the preciousness of water resources. Increase the teaching materials on the water conserving fixtures such as ultra-low flush toilets and low-flow water faucets to let students understand how much water can be saved. Increase the teaching materials on water conservation methods and let students think about the easy to do water saving ideas. Monitor the water bill with students monthly and make this activity a part of water conservation education.

5. Conclusions

This study has established an evaluation framework for elementary school teaching materials for sustainable development, which consists of four teaching scopes, ten learning indicators and 21 learning topics to be used in the framework through literature reviews, expert interviews and the Delphi method. The AHP method was used to identify the relative or absolute weights of the teaching scopes and the learning indicators. The order of weighting shows that, among four teaching scopes, ecology has the highest weight, followed by waste reduction, energy saving and health; among the ten study indicators, the top three are in order of biodiversity, waste reduction, and energy conservation, and the lowest is indoor environment. Using this framework, an elementary school can check if the textbooks adopted by the school cover all the SD teaching materials and design extra teaching materials not covered in the textbooks if necessary. Also, a sustainability scoring table has been developed based on the importance of each scope and indicator in the framework. It can let a school know how much improvement of teaching materials in an indicator is needed. The application of scoring table in the study case demonstrates that the sustainability scoring table is a convenient way to verify the SD teaching materials in a school. As to the SD materials in the current textbooks, the established evaluation framework can be used as a reference to modify them by adding the SD materials needed to make them more complete and suitable for elementary school children.