Investigating Chemistry Teachers’ Assessment Knowledge via a Rubric for Self-Developed Tasks in a Food and Sustainability Context

Abstract

We investigated the competence of in- and pre-service chemistry teachers and teacher mentors in designing sustainability- and systems-oriented online tasks for their students. Using a dedicated rubric, we evaluated their assessment knowledge (AK) as reflected in the tasks they had developed. The rubric is based on four attributes: integration of sustainability and chemistry, diversity of thinking skills, the variety of system aspects, and diversity of visual representations. Implementing a qualitative case study approach, we tracked the professional development of three purposefully sampled teachers in addition to using the rubric to score their tasks. Combining the rubric scorings and the qualitative investigation via feedback questionnaire revealed new insights. Besides the teachers’ content and pedagogical knowledge, the case studies’ context and relevance to the teachers were found central to their ability to assess learning. This research contributes to the theoretical understanding of AK of teachers with different backgrounds and professional experiences. The methodological contribution stems from the analysis of self-developed tasks based on a designated rubric, which should be further validated.

1. Introduction

As the world’s population is expected to reach nearly ten billion people by 2050, sustainability issues are expected to become unprecedentedly pressing and inescapable [1]. To achieve sustainability goals, such as the UN’s Sustainable Development Goals—SDGs [2], education systems must modify teaching and learning methods to better suit the need to explain and foster the understanding of sustainability as a complex, multifaceted concept (An explanation of each abbreviation appearing throughout the article is provided as Supplementary Materials.).

Achieving sustainable development goals can be seriously hampered by lack of appropriate education, which can significantly stimulate it [3]. Both Basheer and colleagues [4] and Tal and colleagues [5] recently reported a low level of knowledge about and understanding of green chemistry and sustainability among pre- and in-service chemistry teachers in Israel, where the current study was conducted. Although a complete alteration in higher education curricula to change this undesirable reality is unlikely, mentor teachers can and should utilize interventions to raise awareness of sustainability issues among their students, who are tomorrow’s teachers, and motivate them to connect science to real-world scenarios [4]. Context-based learning (CBL) can help achieve this goal and enhance the relevance and engagement of student teachers, as well as their future students, as they teach and learn real-world scenarios [6]. This makes CBL particularly important in sustainability education [7].

According to Shulman [8], the professional knowledge of teachers can be divided into several types (knowledge bases, in his words), including content knowledge (CK) and general pedagogical knowledge (PK). Pedagogical content knowledge, PCK [9], as Shulman put it, is a “special amalgam of content (i.e., CK) and pedagogy (i.e., PK) that is uniquely the province of teachers, their own special form of professional understanding” [8], p. 8, thereby distinguishing them from content experts. Another type of knowledge that differentiates science teachers from science experts is assessment knowledge (AK), an aspect that is absent from Shulman’s theory of teachers’ knowledge bases as a separate entity from PCK and was added about a decade later by Magnusson and colleagues [10]. Avargil and colleagues [11] found that only teachers who had already achieved high levels of CK, PK, and PCK were able to demonstrate a developed AK, suggesting that AK is the highest stage in teachers’ professional growth.

As part of their PK, education students should be able to represent complex systems with multiple models, especially with respect to food and sustainability issues, which are often complex in nature [5,12,13,14]. Conceptual modeling in both science and engineering has been shown to facilitate learning and assessment of systems thinking [12]. While conceptual modeling is a standalone discipline [15], the benefits of conceptual modeling span across a wide range of applications.

In the present study, we aimed to investigate the competence of pre- and in-service teachers and mentor teachers in constructing sustainability-oriented online tasks for their future or current students. To this end, the participants were instructed to integrate aspects of sustainability and chemistry into their self-developed tasks. Before starting the development of the assessment task, the participants were required to complete a context-based learning process on conceptual modeling and systems, described in Section 3.3. Our analysis of the self-developed tasks focused on the pre- and in-service teachers’ AK as reflected by (1) knowledge and understanding of sustainability and chemistry, (2) diversity of thinking skills, (3) systems thinking and conceptual modeling, and (4) diversity of representations. We based the AK rubric on Tal and colleagues’ work [5]. Using the case study approach, we documented the entire process that led to the final products—the tasks that the participants developed. This documentation has enabled us to gain insights that the rubric alone cannot reveal.

2. Theoretical Background and Literature Review

This section includes the theoretical background behind the main issue involved in this study: teacher assessment knowledge. Next in the section, the relevant literature will be discussed about education for sustainable development (ESD), context-based learning (CBL), and relevance in learning.

2.1. Teacher Knowledge: Integrating Systems, Sustainability, and Chemistry

Our theoretical framework is based on teachers’ knowledge according to Shulman’s theory [8,16,17], which emphasizes PK, CK, and PCK. As an extension of Shulman’s theory, AK refers to teachers’ ability to design and implement appropriate tasks that assess the knowledge and abilities of their students [5,11,18,19,20]. Magnusson and colleagues [10] considered AK as a component of PCK, as argued also by Friedrichsen and colleagues [21] and Park and Oliver [22]. Others, such as Avargil and colleagues [11], conceptualized AK as a construct that is separate from PCK. Furthermore, they [11] claimed that AK is elevated above PCK, considering it a difficult knowledge challenge and an ability that can only be reached once PCK has been well developed. Schafer and Yezierski [23] argued that research on the process teachers go through to design assessments, especially chemistry-specific assessments at the high school level, has been scarce. Over a decade earlier, Friedrichsen and colleagues [21] made the same argument. Teachers’ professional development programs focused on effective assessment are therefore an important pillar in establishing this advanced knowledge [11,24,25]. Following [11] and others, we treat AK as a construct that is separate from PCK.

In improving our schools, teacher education and the knowledge the teachers acquire play an incredibly important role [26], whether as professional development of in-service teachers [27] or pre-service teacher preparation [28]. However, both pre- and in-service chemistry teachers often lack appropriate skills and knowledge when it comes to sustainability issues and their instruction [4,5,29], including related assessment knowledge. The same is true for teaching systems thinking skills and concepts [30]. According to Clandinin and Connelly [31], a distinction must be made between teacher knowledge and knowledge for teachers. Whereas the former is constructivist in nature and emerges during the teacher’s practice, the latter is the knowledge that is transmitted to teachers from researchers and policymakers rather than being created during the teacher’s routine and experience. While knowledge for teachers is mainly explicit, teacher knowledge is implicit in nature, making its transfer to other contexts more challenging [32]. AK represents an advanced form of knowledge that teachers have about the assessment of their students. To be effective, teachers need to possess AK at a level that enables them to design tasks that are commensurate with the various students in their classes, each having a different level of understanding and skills [11]. We consider AK to be teacher knowledge, i.e., knowledge about assessment that the teacher has, rather than knowledge for teachers, which is the knowledge about assessment that education experts deem necessary for a teacher.

2.2. Sustainable Development and Sustainability Education

Across the globe, humanity faces numerous problems that are interconnected, interdependent, and complex [33]. These issues are contained within, or at least strongly related to, what is known as “sustainable development,” as manifested in the UN’s 17 sustainable development goals (SDGs) [2]. There is tight interdependence within and across the SDGs [34], which is expressed by the 169 associated targets and 232 related “integrated and indivisible” indicators [2] (p. 7). For example, in terms of sustainability, the progression of one system can negatively or positively affect the trajectories of other systems, emphasizing the interdependence between environmental, economic, and social systems [35]. It is also possible for two SDGs that are not interdependent in the short run to become so over the long run, and vice versa [36]. Accordingly, UN SDG-related publications emphasize that countries should adhere to the 2030 Agenda (On 25–27 September 2015, the UN Summit in New York launched the 2030 Agenda for Sustainable Development [2,19].) as a whole rather than treat each SDG separately [2].

Given this grave state of affairs, a systemic perspective must be taken even when tackling each sustainability issue separately. In the case of two or more interrelated sustainability goals, issues, or processes, a systems approach is mandatory [37]. Such an approach is imperative also for teaching and learning sustainability, as outlined in the recommendations of UNESCO [38] for teachers’ competencies needed to effectively engage in ESD. As noted in that document [38], “one of the great challenges of ESD is to have student teachers understand the interrelatedness of the environment, society, and economy and have this interrelatedness be evident in their teaching” (p. 43). Accordingly, among UNESCO’s recommendations for mentor teachers is to require student teachers to take interdisciplinary courses while providing them with appropriate educational and instructional means, and teaching them pedagogical techniques for fostering and cultivating higher order thinking skills.

On the one hand, it is widely acknowledged that chemistry, which offers a perspective for understanding the particulate basis for sustainability [39], plays a central role in achieving sustainability goals [40], and this is true also for chemistry education [41]. On the other hand, even though many chemistry concepts are interdependent and interact with each other, they are often taught and studied in isolation [42]. Directing chemistry education to be integrated into sustainability education requires a shift in both content and pedagogical approaches [43]. Burmeister and Eilks [44] suggested that sustainability should be an independent topic in chemistry teacher education programs. It could be useful to also connect sustainability topics regularly with chemistry-specific topics, as the combination of the two approaches is likely to prove most fruitful [44]. Curricula based on an interdisciplinary approach will enable students to understand the relationships between a wide range of particles, as well as to appreciate the wider impact of chemistry on the economy, society, and the environment—the three facets that make up sustainability in its broader sense. Emphasizing the relevance of sustainability in the economic, social, and environmental contexts is therefore important to establish the study of sustainability in educational systems [3]. Following a systems thinking-oriented intervention, Szozda and colleagues [45] found that undergraduate chemistry students tended to neglect the human impact on chemistry-related climate change issues. However, according to the planetary boundary framework [46,47], the environmental aspect of sustainability is the most decisive one, while the other aspects—the economic, social, and socio-economical aspects—depend on the environmental aspects and need to be regulated accordingly.

2.3. Context-Based and Relevant Learning in Chemistry and Sustainability Education

Context is inherently multifaceted and cross-disciplinary. However, students and teachers alike face difficulties understanding context and relevance in chemistry education [48]. Therefore, many chemistry learners may grow to dislike chemistry and perceive chemistry classes as irrelevant to them [49]. However, some uncertainty surrounds the meaning of “relevance” and what the best ways to achieve that relevance are [50]. In an attempt to clarify this theoretical uncertainty, Stuckey and his colleagues [51] proposed a model of relevance that includes the vocational, social, and individual dimensions.

Curricula for chemical education had traditionally emphasized chemistry as a scientific subject whose study aims to prepare future chemistry professionals. In recent decades, this view of chemistry has gradually changed—general chemistry is studied also by those who do not major in chemistry, and chemistry courses are included also in curricula for preparing professionals other than chemists. This change has led to the adoption of new approaches to teaching chemistry [52], with context-based teaching and learning being one of the changes that uses contexts and applications to boost learners’ interest and, therefore, their understanding [53,54]. Context-based learning (CBL) can increase relevance to learners, as they perceive learning chemistry in context to be more enjoyable and useful [55].

Basing curricula on contextual learning is a potentially key element in solving common problems in teaching and learning science in general and chemistry in particular. These problems include low relevance, content overload, lack of connection between ideas due to learning them as isolated facts, low motivation in science lessons, and the inability to transfer the same knowledge and understanding to new settings or situations [48]. In their systematic review, Bennet and colleagues [56] found evidence to support the claim that CBL does not adversely affect students’ understanding of science concepts, while it significantly motivates them and nurtures their positive attitudes toward science. In a later review [57], Bennet and colleagues argued that while “context-based approaches” are open to a wide range of interpretations, relevance to students’ lives and their present and future interests and careers is a common denominator in interventions that have been reported as successful.

Pilot and Bulte [54], who stressed the importance of understanding how interdisciplinary contexts can facilitate CBL, found that integrating contexts into chemistry curricula can help prevent overload and provide a representative and relevant curriculum, in accordance with other studies [48]. According to Schwartz [58], the formal curriculum for contextual chemistry learning should emphasize, among other things, interdisciplinary connections between society and science, which reflect the sustainability problems and issues society is facing.

Hofstein and colleagues [59] argued that the social dimension in science education is often overlooked. Since this dimension is highly relevant in sustainability [29], ignoring it may severely hamper attempts to make sustainability education more relevant. Pilcher [40] suggested that by engaging in sustainability issues, students who usually see little relevance in chemistry to their lives might find relevance by linking chemical knowledge with society’s pressing needs, thereby making learning more meaningful. Pilot and Bulte [54] stressed the importance of CBL-specific professional development for teachers, particularly in view of the innovative assessment that accompanies CBL.

Scoring rubrics serve as tools for measuring AK levels among learners. Although these tools are quantitative in nature, Panadero and Jonsson [60], who examined rubrics as formative assessment tools, found that beyond the scoring that rubrics produce, they can also support student learning. This study aimed to investigate teachers’ and mentor teachers’ competence in constructing ESD-oriented online tasks for assessing their future or current students’ competencies.

2.4. Research Questions

Given the above research aim, the resulting research questions (RQs) were the following:

(1)

How do chemistry teachers of different backgrounds apply concepts of systems, sustainability, chemistry, and instructional design to create contexed-based food-related assessment tasks?

(2)

How does the rubric capture teachers’ professional development in the following four attributes: sustainability and chemistry, systems; thinking skills, and visual representations?

We anticipated that following a learning process oriented to systems thinking and conceptual modeling, the participants would have a solid basis for developing the assessment tasks using system concepts and conceptual models. Regarding the integration of all the relevant aspects into a coherent assessment task, we expected to find background-dependent differences between the participants. As we did not find studies that directly address these questions, we had no hypotheses as to the effect of the participants’ backgrounds on their self-developed assessment tasks.

3. The Case Study Research Methodology

A key goal of the present study was to investigate chemistry teachers’ and mentor teachers’ readiness and ability to engage in an educational endeavor that calls for implementing their CBL knowledge, systems thinking, and assessment knowledge while developing sustainability-related tasks. To better characterize the process and not only its product—the tasks developed by the participants—we opted for a small and purposeful sample to be examined through the case study approach [61].

The case study approach is most suitable when the researcher wishes to “study a case when it itself is of very special interest” [61] ( p. xi). This approach is expected to catch “the particularity and complexity of a single case” (p. xi), where each case is “a specific, a complex, functioning thing” (p. 2). Following this qualitative view of the case study approach, we adopted the instrumental case study—a particular case that researchers study in an attempt to gain insight into a more general question they are trying to answer. The current study is considered a collective case study—a study encompassing several case studies [61].

3.1. Trustworthiness

From a positivist point of view, reliability, validity, and objectivity are only relevant in quantitative research. Guba [62] established measures that address these issues in qualitative research. Many researchers have adopted Guba’s four-criteria construct [63], and we too have used it to ascertain the trustworthiness of our research, using measures to enhance credibility (instead of the rationalistic internal validity), transferability (instead of external validity), dependability (instead of reliability), and confirmability (instead of objectivity).

Criterion 1: Credibility–the extent to which the findings are congruent with reality. First, we used data gathered from questionnaires that included both open- and closed-ended questions, conceptual models, self-developed tasks, and written feedback to carry out triangulation, and to elaborate and clarify the findings. The qualitative components were each participant’s written feedback, and close observation and documentation of it by one of the authors, while the rubric constituted the quantitative component. Secondly, we included only participants who expressed their willingness to participate in the study and met all the research requirements, i.e., those who completed all the study units and the learning process and submitted a task according to the requirements detailed below. Participants were also encouraged to share their experiences at any point during the research. Thirdly, the first author interacted closely with the research participants during every stage and every aspect of the research, and insights were continuously and iteratively shared with the other researchers. Fourthly, meetings were held frequently throughout the research to share perspectives and experiences that broadened the researchers’ vision. Lastly, to further reduce positive confirmation bias [60], we used a pre-registered study design, in which the research plan is established before data collection. In so doing, bias in data interpretation may be reduced [64].

Criterion 2: Transferability–the extent to which the findings can be applied to other situations. Guba [58] argued that in qualitative research, considerations of transferability are the responsibility of the reader more than that of the researcher. Nevertheless, to ensure that potential readers can make the necessary transfer, researchers must provide sufficient contextual information. To make the necessary transfer to other settings, a rich, detailed, “thick description” [65] of the research context is provided throughout the article.

Criterion 3: Dependability–the extent to which the study can be repeated by further research, even if identical findings will not necessarily be obtained. Demonstrating credibility implies realizing dependability, but Shenton [63], following Guba [62], advocated considering dependability more directly. To directly address dependability, we provide a detailed account of the decisions that were made throughout the research, to allow future researchers to reproduce the research process even if the findings may differ. For example, we present an explanation for the inclusion of the learning process before starting the development process, and a rationale for choosing food production to contextualize the learning process and the task development.

Criterion 4: Confirmability–the extent to which the findings are the result of the experiences and ideas of the participants, rather than researcher predispositions. First, to reduce the effect of investigator bias, the three authors took an active part throughout the study, each bringing their own perspectives and predispositions to the study, balancing each other out. Secondly, multiple data sources have been used to perform both methodological and data triangulations. Thirdly, a continuous written record of introspections and possible orientation shifts was kept throughout the research.

3.2. Research Context and Participants

We used critical case sampling, which is a purposive sampling method, in which researchers aim to deepen their understanding of the phenomenon being examined by selecting critical cases deemed important to the study. This guideline enabled obtaining a diverse set of critical cases within the sample [66]. Thirteen STEM teachers were recruited at the Faculty of Education in Science and Technology at the Technion, Israel Institute of Technology, of whom seven were females. Five of them had an academic background in chemistry or chemistry-related domains (e.g., chemical engineering), while the rest had a more diverse background in science or engineering fields other than chemistry. The participants included 10 pre- and in-service teachers, who were undergraduate and graduate students at the Faculty of Education in Science and Technology. Mentor teachers (the remaining three) were participants who engaged in teacher education—prospective teachers training or in-service teachers’ professional development. All the participants took part in the study either as part of a research project that grants academic credits or as part of a professional development course for in-service teachers.

3.3. The Intervention

In a previous study [12], we found that most pre- and in-service STEM teachers and mentor teachers are not familiar with system concepts, such as function, structure, purpose, and behavior, in line with other studies, such as Arnold and Wade [30] and Yoon and colleagues [67]. We, therefore, had to lay the foundations for systems thinking and conceptual modeling for the participants, so that they can later create tasks. The participants completed this learning process at their own pace as part of an elective course, following which they began to develop their task, focusing on a food production topic of their choice, with aspects of sustainability. Issues related to food are interdisciplinary and complex in nature [68,69], and relevant to all of us. Accordingly, and based on our previous research [12], we assumed that modeling food topics can be used to develop and evaluate system thinking and conceptual modeling skills.

The participants were asked to make the task suitable for high school students and incorporate into it at least one conceptual model that focuses on food and sustainability contexts as a reference for answering questions in the task. The task called for integrating diverse multimedia representations, such as text, videos, charts, and figures, followed by questions of various kinds and difficulty levels.

3.4. Research Procedure, Tools, and Data Analysis

We investigated and characterized the challenges that participants faced, the difficulties they coped with, and the contributions they gained during the online learning process and task development. The task that the participants developed served as a tool to evaluate their ability to integrate chemistry, food, and sustainability issues, conceptual modeling, and systems concepts into one coherent online learning unit. At the end of the research, the participants responded to a feedback questionnaire, which was analyzed to gain additional insights.

3.4.1. The Online Learning Process

Object-Process Methodology–OPM, is a formal model-based systems engineering methodology that is bimodal (including both text and graphics) and is simple to learn. Its simplicity is expressed in the use of three types of entities: objects—things that exist, processes—things that transform objects, and states—situations objects can be at during their lifecycles. In OPM models, the structure and behavior of a system are combined in the same, single kind of diagram—Object-Process Diagram (OPD)—to express the desired function, or in the case of natural phenomena—the expected outcome. OPM is expressed bimodally in graphics and text. Each graphical representation, OPD, created in OPCloud [70], a collaborative, cloud-based OPM modeling environment, automatically generates a textual one, Object-Process Language (OPL), in a subset of English or another natural language [71].

To prepare the participants for developing their tasks, we developed four food-related online assignments containing short videos, photos, text segments, and OPM models, with each assignment consisting of a questionnaire and a quiz. The first assignment, for example, was in the context of chocolate production, followed by a quiz on cod fish processing. The content of the entire learning process and an example from the third online assignment are presented in

Table 1. The content of the four online assignments that formed the learning process.

Assignment #	Content
1	- Introducing OPM and basic system concepts. - Identifying objects, processes, and states in a system.
2	- System aspects: function, structure, and behavior. - Structural relations, state transitions, system aspects, and OPM modalities: OPD and OPL.
3	Understanding the System Diagram (SD): System purpose—beneficiary and benefit, system function; and process enablers—agents and instruments.
4	- The first detail level (SD1) of the OPM system diagram, divided into major subprocesses. - Synchronous vs. asynchronous processes.

and Figure 1, respectively. This learning process is described in more detail by Akiri and colleagues [12], who concluded that this process is useful in endowing students with the basics of systems thinking and conceptual modeling in a relatively short intervention. We applied this finding in the current research.

3.4.2. The Task Development Process

After the 13 participants became acquainted with systems thinking and conceptual modeling in the learning process, they were asked to develop a task in the same style as the tasks assigned to them during the learning process. We did not provide more specific instructions in order to be able to evaluate the participants’ AK from different perspectives. The tasks were submitted via email as a text file (DOC or PDF), containing a link to their Google Forms online task, their complete OPM model, suggested answers to all questions included in the task, and answers to the feedback questions. After further processing, the tasks they had created can form a short learning unit, for their own use or others.

We analyzed the tasks developed during the study using an assessment knowledge (AK) rubric presented in

Table 2. The assessment knowledge rubric for teachers’ self-developed tasks–A total of 10 points.

Scoring Attribute	1 Point	2 Points	3 Points
Chemistry and sustainability	Negligible reference to both chemistry and sustainability	At least one aspect of chemistry or sustainability is integrated into the task, but not both	At least one aspect of chemistry and one aspect of sustainability are integrated into the task
Diversity of thinking skills	Only one type of thinking skill is included	Two to three types of thinking skills are included (e.g., understanding, comparing and applying)	Four or more types of thinking skills are included, with at least one activity requiring the respondent to demonstrate novelty
Variety of system aspects	The task involves using basic level system aspects and activities	The task involves using advanced level system aspects and activities	The task includes a requirement to design an OPM diagram from scratch
Diversity of visual representations * (bonus point)	Four or more visual representations of the following—text, tables, figures, diagrams, videos, hyperlinks, and original content	-	-

* A representation that did not contribute to the task or could not be deciphered, such as a poor-quality image or an irrelevant video, was not counted in the number of representations. Links to the videos that were included in the chosen cases are provided as Supplementary Materials.

, which we had developed based on [5,72]. The rubric consists of four attributes: integration of chemistry and sustainability, diversity of thinking skills, variety of system aspects, and diversity of visual representations in the task. It is worth noting that the participants were not exposed to the rubric before or during the learning process and the task development, so it was not part of the intervention.

For the sake of clarification, we note that “novelty”, which is required to be integrated into the task to achieve a full score in the attribute “diversity of thinking skills”, means designing models from scratch, and was formerly known as “creating” [73] in the thinking skills of the revised Bloom’s taxonomy.

3.4.3. Written Feedback

As feedback on their task development process, the participants were asked to answer open questions such as what did you like about the process? what impact did the learning process have on your development process? and what were the main difficulties in implementing the conceptual modeling in OPM in the task you developed? There was a total of eight questions, the answers to which were attached to the final task submission file. The qualitative data obtained from the responses were added to the documentation of the correspondence between the participants and the authors. A content analysis was then conducted to extract key themes from the data collected. The themes that emerged from the analysis enhanced other findings and inspired the research conclusions [74].

3.5. Ethics

Obtaining informed consent and ensuring confidentiality were the two most important ethical considerations in this study. At the beginning of the study, all participants were required to fill in an informed consent form. The study was approved by the Behavioral Sciences Research Ethics Committee of the Technion—Israel Institute of Technology, Approval #2020–165.

4. Findings

In what follows, we discuss three case studies composed by Benny, Leila, and Dan (pseudonyms), as they represented three different backgrounds related to chemistry teaching and different task characteristics while also sharing similarities. In all three cases discussed below, the tasks were submitted both on the Google Forms platform and as a text file with answers to the feedback questions. We chose the three cases before analyzing the data, as they share similarities such as teaching chemistry, yet the participants’ experience in chemistry teaching was diverse, as are their backgrounds. One participant was an experienced in-service teacher, a second was a practical engineering teacher, and a third was a pre-service teacher.

4.1. Case 1—Experienced Chemistry Teacher and Teacher Educator: Benny

Benny is a high school chemistry teacher with almost 30 years of teaching experience. His academic background includes bachelor’s and master’s degrees in chemistry and chemical education, respectively. In parallel with his work as a chemistry teacher, he serves as a mentor and instructor for new chemistry teachers. He developed his task as part of a professional development course, which took him about a month and a half to complete, after completing the learning process in about three weeks.

His task, presented in Figure 2, focused on wine production and included diverse agricultural, agronomic, engineering, chemical, biochemical, and historical aspects. He explained that the choice of the topic for his task stemmed from a personal acquaintance with the wine world due to the occupation of one of his family members in this field. The central background story began with the importance of growing and harvesting the wine grapes for the quality of the wine, long before arriving at the winery, with a focus on the optimal harvest timing. Accordingly, the questions incorporated later in the task dealt with various aspects of growing and harvesting quality wine grapes. The integration of the above aspects of wine production into the task and linking them to sustainability and chemistry, and their weaving within a systems and modeling instructional framework, were done at a high level.

His task began with a presentation of OPM through three videos to establish a general understanding of conceptual modeling and OPM. A textual description of grape harvesting was followed by a detailed OPD, and questions of various kinds related to this diagram. A modeling assignment related to the topic taught in the task appeared at the end of the task.

The sustainability task aspect was wastewater treatment in the wine industry, integrated with the chemical oxygen demand (COD) index—an indication of wastewater toxicity. While not included in the OPM model, these sustainability and chemistry aspects were linked to it in a relevant way, with questions that reflected wine production as the contextual common thread, requiring different thinking skills to respond.

During the task development, Benny and the first author were in contact regarding questions related to modeling the wine production process and clarifications regarding the integration of sustainability into the task. In his written feedback, Benny noted that composing the task required a deeper understanding of modeling and system concepts than that needed in the learning process:

“The aforementioned form of learning, in addition to the fact that it brought me to the task development stage with a high level of readiness, produced study patterns for me, which I used in the development of the task. For example, before building an OPD for the system I chose, I rewatched the videos dealing with the subject of system characterization: purpose, benefit, etc., in order to further refine my understanding. The process of developing the task was for me a self-examination of what I learned from the online learning units”.

This insight is consistent with the fact that some help was needed during the task development, unlike the learning process he performed without any help.

In Benny’s view, a task of the type he developed “can be harnessed to present complex processes in the study and assessment of natural sciences and for artificial technological systems.” He qualified this statement, thinking that “the method in the scope in which we learned it by ourselves is quite complex for high school students […]. It is possible, however, that with adjustments of the content it will be possible to create a process that can be more easily adapted to high school or even middle school students.” He was well aware of the modifications required to move from a learning unit intended for higher-education students and content experts to one that would be fit for school students.

4.2. Case 2—A Technology College Chemistry Instructor: Leila

Leila holds a PhD in pharmacological chemistry and is currently a teacher at a technology-oriented college (equivalent to a two-year college) for practical engineers, teaching general and organic chemistry. In addition, she holds a B.Sc. in Chemistry Education. She has more than ten years of teaching experience, during which she earned a chemistry teaching certificate that includes practical experience in a secondary school and education studies. She carried out the learning process and developed her task as part of a professional development course. It took her about two weeks to complete the learning process, and about one month to develop the task.

Similar to Benny’s, Leila’s task opened by presenting OPM through videos to establish a basic understanding of conceptual modeling with OPM. During the learning process, the first author was in contact with her regarding modeling-related questions, but during the task development, she did not have questions.

Regarding relevance and context, Leila felt that the learning process influenced her in choosing olive oil production, “a topic relevant to today’s life [which is] interesting and complex, with ethical, technological, sustainability, and environmental aspects”. Indeed, olive oil is a basic ingredient in Israeli cuisine and Middle Eastern cuisines, and the olive tree is very common in the countries’ landscapes. After presenting the basics of OPM modeling as they appeared in the assignments that made up the learning process, she introduced olive oil production from an economic and geographical perspective. At the end of the introduction, the by-products of olive oil production were briefly mentioned—olive mill wastewater and solid waste. Following the introduction, various engineering aspects of olive oil production were discussed throughout the task accompanied by additional information and relevant questions.

Her task has ample, carefully selected, and highly relevant flowcharts, figures, photos, illustrations, videos, tables, and conceptual models. Sustainability was included in the OPM model within the task, but not as an integral part pedagogically: no question referred to the OPM model (Figure 3), and there was no need to extract information from it regarding the sustainability aspect—olive mill wastewater. As shown in Figure 4, two points were taken off her task grade since the combination of sustainability and chemistry could have been more integrative and more prominent.

The chemistry aspect was neither an integral part of the task nor was it linked to sustainability. Rather, the chemistry in this task related to the nutritional value of olive oil and the oxidation processes of olive oil during its production, and as in the case of sustainability, chemistry did not play a substantial role in answering the questions successfully.

Leila explained that the context of her task was selected based on her own first-hand experience and knowledge, rather than choosing just any food-related context: “The process is challenging, especially the task development which requires thinking and learning skills at remarkably high levels. However, the diversity of visualizations and demonstrations of the problems and the systems taken from everyday life experiences [...] helped the understanding [in developing the task].” The diverse multimedia in her task was also influenced by the learning process that included diverse representations: “[After completing the learning process, I used] videos, illustrations, links to literature and information resources to find relevant information to the topic under discussion and combine this information in the task to create a more coherent learning sequence”.

The learning process, in Leila’s experience, contributed to top-down thinking by allowing “a broad view of the big picture of the system/problem, and after that slowly diving into more details to present it in the most accurate way, to find a solution and to understand how it works down to the smallest details that make up the whole.” In her opinion, such a learning process holds great promise to science educators, provided it is accompanied by “establishing and implementing relevant training programs for teachers and then for students, and creating a database of ready-to-use assignments that can be easily tailored to meet specific needs”.

4.3. Case 3—Pre-Service Teacher and Chemical Engineering Student: Dan

Dan is a pre-service chemistry teacher who is concurrently in his senior year in chemical engineering and the chemistry teaching certificate track. He completed the learning process and task development as part of a research project that grants academic credits. His task on sweet peppers processing took him much longer to develop than the other participants–about five months, during the academic year. The learning process also took him longer than others to complete, about two months.

Dan explained that he chose this topic because family members are involved in this field, so he felt familiar with and committed to the process. Dan needed help and advice in developing the task in both designing the conceptual models and the instructional-evaluative aspects, such as how to combine sustainability with chemistry and how to integrate the elements into a cohesive whole. The first author had several meetings with him on these issues. His task (Figure 5) focused on the sorting and packing process after the sweet peppers are harvested, and he used OPM almost exclusively rather than creating a broader context. The sorting and packaging of sweet peppers were in the background but lacked an integrative context. The OPM concepts included in the task revolved around basic thinking skills, such as comprehension or application, rarely requiring higher order thinking, such as analysis or novelty. These findings were well reflected in Dan’s feedback: ”The main difficulty was indeed in the integration of the various required aspects and the content to cover key issues. The difficulty also stemmed from the uncertainty about what level or depth to reach in the content: if the level of the topic and content is too high, this may take the focus off the modeling, thereby missing the goal of studying modeling”.

Referring to the learning process, Dan stated that performing it gave him “ideas of how to organize and frame the material [in developing his task]. In addition, by doing so [completing the learning process], I could find ways to assess the potential learners and their understanding.” However, in his case, the diversity within this framework was limited, both in thinking skills and content. His perception of the learning process’s contribution to the development of his task differed from the actual contribution. According to his feedback, he achieved awareness of AK principles, but this awareness was not reflected in his product—the task he developed.

Regarding the relevance and context of this learning process, Dan wrote that “The important thing that is required for such an integration [of the learning units focused on chemistry, systems, and sustainability] is learning units through which the students will learn the content, but such that will adapt the explanations and instructions to the language, content, examples, and topics relevant to the students.” He also stressed the importance of presenting students with “something relevant to their daily lives, which they will be able to relate to.” He added that the same applies to the teachers who will present the material so that the relevance is reflected in their teaching.

5. Discussion

Education plays an important role in pursuing sustainability. Providing the molecular, microscopic, and macroscopic levels of chemistry, sustainability, and chemistry education are of crucial importance in achieving sustainability goals. This is especially true when sustainability is examined and taught through the lens of systems thinking rather than just through the more prevalent reductive lens [42]. The qualitative case study approach we used is particularly suitable for capturing the breadth of the process participants went through during the research. We were interested in both the commonality and the uniqueness of each case compared with others [61].

This study aimed to characterize online learning units—tasks that pre- and in-service chemistry teachers and mentor teachers developed and assess their competence to engage in such activity. Developing the tasks was the final assignment of a conceptual modeling-oriented online learning on food production. To assess the participants’ assessment knowledge, we developed an AK rubric for the specific combination of systems, sustainability, and chemistry. The findings from three case studies, which we chose to present are discussed below with reference to relevant literature. Burmeister and Eilks [44] reported that most German in-service teachers lack deep knowledge of sustainability and sustainability education, and that teacher education does not prioritize sustainability in chemistry education. These results are in line with studies conducted in Israel [4,5]. Even when teachers understand and teach a certain sustainability topic, assessing their students’ understanding is often still difficult. Assessment knowledge, as a higher form of knowledge than PCK [11], requires highly domain-specific knowledge alongside uniform pedagogical principles, such as multiple representations and thinking levels. In this study, we examined the process that chemistry teachers and mentors go through when assessing the understanding and knowledge of food production issues using a system perspective with reference to chemistry and sustainability.

The first notable insight refers to the participants’ choice of topic. In all three cases, the participants chose a topic that was close to them in some way. As Clandinin and Connelly [31] claimed, knowledge is often not an objective thing that can be transmitted by and learned from the teacher; rather, it arises from the teachers’ “personal practical knowledge” [p. 74]. This claim is consistent with our findings. Given a free choice of topic and instructional design, prospective and experienced teachers and mentors with various backgrounds chose a topic in which they had first-hand knowledge or personal background. Usually, the topic to be learned is predefined by the curriculum, and the teachers have limited instructional options to teach it, leaving less-than-desired room for personal expression, and often less engagement in teaching it.

Our findings suggest that context plays a key role in teachers’ knowledge and practice, both as a challenge and as an opportunity. On the one hand, based on the feedback from the three case studies, the relevant context allowed the teachers to delve into and be committed to the task development process. On the other hand, from the documentation of the interactions between the researchers and the participants during the research, a picture emerges of lack of clarity as to how to combine different topics and skills within the same chosen context, specifically, combining sustainability, chemistry, and food production. Our findings may also suggest that teaching and assessing based on context alone may not be sufficient [57]. A context that is not relevant or disconnected from personal experiences may be as advantageous as traditional learning that is not context-based. A context relevant to students is very welcome and has often been proven to be important in learning [55]. Yet, we believe that our research also emphasizes the importance of relevance to teachers, especially in complex and demanding teaching and assessment practices. The ability to make stronger claims in this regard depends on substantial further research, as discussed in Section 5.1.

The learning process in this research simulated a teacher professional development program. Subsequent implementation of the knowledge and skills that the participants had gained took place in task development. Some of the desired skills and knowledge, such as conceptual modeling and system thinking, were explicitly instructed, while others, such as relevance, context, instructional design, diversity, and integration were applied implicitly. This transition from implicit learning to the explicit application was exemplified in the case studies presented earlier. Instructional principles applied in the developed tasks were perceived by the participants as being promoted by the learning process, even when they were not explicitly taught.

The second insight concerns the AK-CK relations in sustainability education and science education in general. Although this study aimed to examine the teachers’ AK rather than their CK, in all three cases, we observed that despite their basic sustainability knowledge, participants encountered difficulties while integrating sustainability into the tasks they developed. Although Benny, Leila, and Dan had different educational backgrounds and experiences, they all faced difficulties with how to incorporate sustainability into their assessment tasks. They needed consultation and guidance on how to integrate the sustainability aspect, and to a lesser extent—the chemistry aspect. This finding is in agreement with the claim of Avargil and colleagues [11], which places AK at the top of the teachers’ CK, PK, and PCK knowledge hierarchy. Second, as suggested by Park and Oliver [22], a deep understanding of the content to be taught by both in- and pre-service teachers can also help with assessment, as students’ misconceptions and mistakes can be more easily recognized. Deep CK enables teachers to continuously assess their students during classes, reducing the need for complex assessment tools. Another aspect that emerged from our three case studies is the assessment flexibility that teachers may have if they have deep CK. Better acquaintance and awareness of various sustainability issues, from chemical aspects to social ones, would have allowed the three task developers to integrate related sustainability issues into their respective contexts more easily. They could combine different sustainability aspects into their tasks, rendering them more realistic and holistic, as recommended by sustainability educators [38,42]. In all three tasks, mainly environmental aspects were discussed, while other sustainability-related aspects such as social and economic ones were glaringly missing. The task developers related to the narrow sense of sustainability, compared to a broad one that includes also economic and social aspects [3]. Our findings in this regard are consistent with other studies on chemistry students and chemistry teachers [45,59], which found that the human and social factors were overlooked as the focus was on more tangible factors, such as pollution by the emission of toxic substances into the environment. Social factors affect the environment even at the whole-planet level [47]. Therefore, teachers’ knowledge about the environment is critical so they can educate the future generation. Indeed, our findings, like others [4,5], emphasize the need to expand chemistry teachers’ sustainability CK besides strengthening their understanding of chemistry-related sustainability issues.

The third insight is related to the effect of teaching experience on assessment knowledge. The three cases we analyzed may illustrate the teaching experience factor, as seen from the final product and also from the documentation of the process itself. The most experienced teacher achieved a better product—the task he developed, in a reflective, regulated, structured, and efficient manner. A deep understanding of assessment practices and principles can help teachers evaluate the learning and progress of their students accurately and effectively [19]. Such understanding includes how to design and implement effective assessment, use data to inform instruction, ask meaningful, diverse questions, and provide meaningful feedback to students to help them improve [10]. Teaching experience may also help teachers develop their assessment knowledge and skills [18]. As teachers gain experience in the classroom, they are exposed to a variety of assessment practices and approaches, which can encourage them to try different strategies and see what works best for their students in various contexts [72]. Through this process, teachers can develop a deeper understanding of assessment and how to use it effectively in their teaching, bearing in mind that AK is intricately connected to PCK [11]. A quantitative study and additional research environments will be necessary to evaluate this claim.

AK and teaching experience are related in terms of teachers’ professional development, as experienced teachers perceive it differently and tend to take different professional courses compared with beginning teachers [24]. Extensive, high-quality formal education may stimulate interest in further education and training to develop skills acquired during formal education. As discussed in the “Limitations and Further Research” section, experience and formal education may be important to the development of teachers’ AK, both directly and through professional development programs.

The specific AK rubric we developed based on [5], integrates instructional design, systems perspective, chemistry, and sustainability knowledge. Applying this rubric highlighted the different backgrounds of the three case study participants. These differences were evident also from the analysis of the development process and the documentation using the case study approach, indicating that the rubric may be valid and can be used in future research. Benny, the teacher mentor, received a perfect 10/10 score. Throughout the development process, he indeed demonstrated high awareness of assessment, content, systems, and representation diversity aspects, reflecting on each step along the way. Leila also showed awareness of these aspects, but her ability to apply this in her task was lower than Benny’s, as reflected by her lower score, 8/10, which was still high, and by a less structured and more exploratory development process than Benny’s. Leila’s lower ability to integrate all the aspects into the task was her main drawback, and this could be attributed to her limited experience in developing new tasks. Task development is not taught in teacher education and most of the teachers are using tasks that are common to others. Dan, the pre-service teacher, encountered many difficulties and obstacles during the development of the task and needed significant assistance and advice along the way. While taking the process seriously, he tended to spend too much time on marginal issues, leaving too little time for adequate integration and diversification of thinking skills taught within the task, as reflected by the score of 6/10 he received.

Issues that deserve special attention have been highlighted in this research, calling for further complementary follow-up studies. The combination of the AK rubric and the qualitative analysis of the case studies offers directions for further research, as discussed in the next subsection.

5.1. Limitations and Further Research

We sought transferability rather than generalizability in our three case studies presentation. Transferability allows readers to refer to the details in the case studies and compare them to those of an environment or situation they are familiar with. By providing a detailed description of the participants’ perceptions, experiences, and contexts, others can evaluate whether the conclusions drawn can be applied to other circumstances, times, and settings. It is, however, likely that further quantitative research involving a larger sample size, diverse teacher backgrounds such as formal education, teaching experience, and different educational institutions rather than just ours, would yield generalizable results that may have broader applications. Our understanding of teachers’ and mentor teachers’ AK in integrating systems thinking, modeling, chemistry, sustainability, and online learning may significantly benefit from such research. Conducting a similar study as part of a professional development or training program with more participants will ensure a higher, more homogenous level of participants’ engagement, yielding a larger, more representative pool of cases for analysis and possible quantitative follow-up studies.

In further research, it would be useful to analyze sustainability- and systems-oriented assessment tasks following context-based and sustainability-oriented training and enhancing CK and PCK aspects and see if the AK will also improve. The relevance of the context for teachers in CBL (as opposed to exclusively learner-oriented relevance) and its contribution to the desired educational goals should also be subjected to more research.

The assessment rubric presented in this article was developed iteratively after repeated revisions during the research, based on a previous study by Tal and colleagues [5]. However, it still requires considerable further research. This rubric was tested on a small sample, and these are only the first results. Increasing the rubric’s resolution by expanding the scoring range for each attribute and validating it in quantitative research will increase its sensitivity and make it more powerful.

5.2. Research Contribution

Our research has theoretical, practical, and methodological contributions. By analyzing the process that the teachers went through from the online learning process to the task development process, we were able to trace the challenges, benefits, and barriers that would not have been revealed if we had only examined the final product—the self-developed tasks. As an example, all three cases analyzed revealed the same insight: a context isolated from experience or personal knowledge is not the immediate choice, and without clear relevance, the context may have a reduced effect. This finding augments our understanding of the way teachers and mentor teachers approach complex assessment processes, such as those presented in this article. By demonstrating different complexions of assessment knowledge based on the teachers’ professional experience, our study contributes to the establishment and expansion of theoretical knowledge in the assessment and development of AK. A practical contribution that stems from the theoretical one is the integration and application of mentor teachers’ theoretical knowledge in tasks they developed for training their student teachers. In contrast to the subjective nature of the qualitative case study approach, our proposed AK rubric reduces the subjectivity of the assessment. This combination enables a more accurate assessment of teachers’ AK levels and the effectiveness of interventions that combine chemistry, sustainability, and systems thinking through conceptual modeling. Effective development and accurate analysis of teachers’ AK are likely to be of great importance in the universal pursuit of sustainability through education for sustainable development. Based on experience gained in this study, developing tasks by pre- and in-service teachers is an effective way to facilitate their engagement with and awareness of AK in general and assessment of systems thinking, modeling, and sustainability in particular.