Evidence-Based Design of a Teacher Professional Development Program for Differentiated Instruction: A Whole-Task Approach

Abstract

Teachers are urged to provide differentiated instruction (DI), that is, deliberately adapting instruction to the learning needs of all students. DI can positively affect students’ academic achievement and their social and emotional development, as well as foster teacher job satisfaction. However, international research, as well as research in the Dutch context, has shown that teachers feel unprepared to provide DI. Hence, the development of teacher professional development (TPD) programs is necessary. In the current paper, the design of a TPD intervention to support DI is presented, in which the content and design approach were deliberately chosen. The intervention content was based on an analysis of the skills and knowledge expert teachers use when providing DI. The design of the intervention was based on the whole-task approach from the 4C/ID model to promote the transfer of learning, among other things. Based on the experiences of the teachers participating in the training pilot (n = 4), we provide our recommendations for future TPD for DI. The next step will be to study the effects of this TPD program on a larger scale to obtain insight into what design characteristics do or do not work, which can be used to further improve this TPD intervention.

1. Introduction

Teachers are urged to provide differentiated instruction (DI), that is, the deliberate adaptation of the content, learning process, assessment product, learning environment, and learning time to the learning needs of all students in a classroom, in order to offer them the best learning opportunities (Smale-Jacobse et al., 2019; Tomlinson et al., 2003) [1,2]. Research has shown that high-quality DI can have a positive influence on student achievement [1,3]. There are also indications that DI can positively affect nonacademic outcomes, such as students’ emotional and social development (i.e., school wellbeing, social inclusion, and academic self-concept [4]). Besides effects on the student level, DI can also have a positive effect on teachers’ job satisfaction (e.g., they feel successful and satisfied in their teaching [5]).

However, a recent international study comparing DI in several countries [6] found that DI is observed in classrooms less often than other teaching strategies, such as classroom management or providing a good learning climate, and that the observed quality of DI is low: “Differentiated Instruction appears to be one of the most difficult skills to implement even for experienced teachers internationally” [6] (p. 283). Teachers feel ill-prepared and feel a lack of support in terms of translating learned DI theory into their own classroom practices [5].

In line with these international findings, a survey study among 927 novice secondary school teachers in The Netherlands revealed that about two-thirds of them did not feel prepared to adapt the (explanation of) subject matter for low-performing (64%) or high-performing (63%) students [7]. A recent observational study [8,9], where 946 lessons at 172 secondary schools were observed by school inspectors, showed that in 59.4% of the lessons, teachers adapted their instruction in line with student progress. Other observed aspects of classroom instruction related to providing DI (as identified by van Geel et al. [10]) were providing challenging assignments (observed in 16.5% of the lessons), clearly sharing the lesson goal (59.1%), activating prior knowledge (61%), and summarizing what students had learned at the end of the lesson (34.3%). It could be concluded that teacher professional development (TPD) is evidently necessary to equip teachers with the competences for DI [5,11].

In this study, we report the development of and blueprint for a TPD program to support secondary education mathematics teachers in developing the competencies for DI.

2. The Effects of Teacher Support for Differentiated Instruction in Secondary Education Are Mixed

Multiple researchers have developed and evaluated TPD programs focused on DI. Smets et al. [12] used an action research methodology to study how three teams of Belgian secondary school teachers provided DI in their classrooms after following a TPD program aimed at improving DI skills. They found that teachers find it hard to assess differences between students’ differences. Teachers also found that when providing DI, not only students’ readiness is important but other student characteristics as well, such as student interest and/or students’ learning profiles. Students’ ability to self-regulate (e.g., is a student able to decide whether they should participate in the extended instruction or not) appeared to be one of the key factors for providing DI. Smale-Jacobse et al. [1] recommend for TPD interventions to include a focus on both subject matter content and pedagogical knowledge, as teachers are required to have comprehensive knowledge of the content and an array of pedagogical and didactic skills to provide DI. Porta and Todd [13] found that in secondary education, there is a “need for existing teachers to engage in ongoing professional development by engaging with other DI professionals and working together collaboratively to share best practices” (p. 582).

In their meta-analysis of studies on the effects of TPD interventions for DI, Kahmann et al. [11] included a total of 27 studies that were (1) published between 2000–2020; (2) written in English, German, or Dutch; (3) empirical, quantitative studies with an experimental design; and (4) for preschool to K-12 teachers. They found a statistically significant medium effect overall (Hedges’ g = 0.55) of the TPD programs on teacher outcomes: teacher classroom practice (Hedges’ g = 0.63), teacher knowledge (Hedges’ g = 0.59), and teacher attitudes (Hedges’ g = 0.16). However, no statistically significant effect of the TPD interventions on student achievement was found.

Of the 27 studies included by Kahmann et al. [11], only 5 took place solely in secondary education [14,15,16,17,18] and 3 [19,20,21] in both secondary and primary education. All these studies, apart from [15], included effects on teacher practice. An overview of these seven studies is presented in

Table 1. Overview of studies in secondary education that measured effects on teachers included by Kahmann et al. [11] in their meta-analysis of professional development programs for differentiated instruction.

Reference	Country	Context	Subject	TPD DI Approach *	Outcome(s)
Hornstra et al. [19]	The Netherlands	Secondary and primary	Not specific	Specific	Teacher practice, teacher attitudes
Hughes et al. [20]	USA	Secondary and primary	Not specific	Generic	Teacher practice
Lange [14]	Cameroon	Secondary	Not specific	Generic	Teacher practice
Schipper et al. [16]	The Netherlands	Secondary	Not specific	Generic	Teacher practice, teacher attitudes
Schipper et al. [17]	The Netherlands	Secondary	Not specific	Generic	Teacher practice
van de Pol et al. [18]	The Netherlands	Secondary	Social studies	Specific	Teacher practice
Vogt & Rogalla [21]	Switzerland	Secondary and primary	Science	Generic	Teacher practice

* The approach of DI in TPD interventions is either more specific (e.g., scaffolding) or generic (e.g., addressing pupils’ different educational needs).

.

In secondary education, the effects of TPD addressing DI at the teacher level are mixed. Several of the studies on DI in secondary education included by Kahmann [11] reported an intervention effect on teacher behavior in the actual classroom. In particular, van de Pol et al. [18] found that the intervention had a statistically significant positive effect on the quality of teachers’ scaffolding. Vogt and Rogalla [21] reported that the TPD program in their study led to teachers significantly improving their adaptive planning; however, adaptive implementation, which was measured with a video test, did not change significantly. The latter is in line with the findings of Hornstra et al. [19] and Schipper et al. [16,17], who also found that teachers did not teach differently in the classroom after participating in a TPD program for DI. Hughes et al. [20] even found that comparison schools (i.e., schools that did not receive any treatment) reported more improvement with respect to DI than the full treatment group.

The two studies in which significant effects were found are subject-specific (social studies and science). In line with these findings, as well as from results in primary education, Kahmann et al. [11] recommend connecting TPD to a specific subject domain. It is unclear whether subject specificity or other contextual aspects (e.g., teacher samples, the support teachers received in their schools, or the intervention content and approach) explain the mixed intervention effects on teaching practice, as much information about this is missing in the publications. Also, the studies differ much in the way they operationalize DI. For Hughes et al. [20], who studied the effects of “culturally responsive teaching” on teachers, DI was only a small part of the study. Some studies approached DI in a generic way, such as the study by Schipper et al. [16], who included TPD with lesson study cycles that “selected objectives that relate to addressing pupils’ different educational needs” (p. 112), and Vogt and Rogalla [21], who studied adaptive teaching competency, which “… consists of the four dimensions: subject knowledge, diagnosis, teaching methods and classroom management” (p. 1052). Another study described TPD interventions that focused on a more specific part or type of DI, for example, van de Pol et al. [18], who examined the effects of a TPD program focused on scaffolding teacher practice.

It can be concluded that there is a lot of variation in how DI is approached. While a definition of DI is provided in most studies, a concrete operationalization and a thorough description of the TPD content (i.e., what the teachers are supposed to learn about DI) is often lacking. Besides the TPD content, the studies often also fail to include a clear explanation of how the TPD method used (how teachers are supported in learning about DI) is supposed to improve teacher skills and student achievement, that is, a “theory of improvement” [11,22].

The goal of the current paper is (1) to thoroughly describe the deliberately chosen TPD design approach and content of our TPD intervention aimed at improving secondary school teachers’ differentiation skills and (2) to conduct a first exploration of how participant teachers experienced that TPD intervention in a pilot study. We chose a whole-task approach, recommended for designing a trajectory for learning complex professional skills [23]. This approach requires having a thorough understanding of the knowledge and skills of expert professionals who have mastered the professional skills to be learned (in our case, differentiating instruction). We therefore used the study of Meutstege et al. [24] for the content of the TPD program, who define DI as “the deliberate adaptation of teaching and learning activities to the learning needs of the learners.” (p. 3). They conducted a cognitive task analysis (CTA) of instructional differentiation by secondary school teachers to obtain an in-depth picture of what competences they require to provide DI.

3. The Required Skills and Knowledge to Provide DI in Secondary Education

The CTA by Meutstege et al. [24] consisted of classroom observations and stimulated-recall interviews (i.e., teachers are stimulated to recall their in-class decision-making process by watching a video clip of their own teaching [25]) with secondary school teachers who are already experts in providing DI, as well as expert sessions with both expert teachers and other DI experts (such as researchers with expertise in DI and educational consultants) to identify the skills and knowledge needed to provide DI in secondary education and the factors that influence the complexity of providing DI (which were identified to support the teachers).

The skills hierarchy (see Figure 1) based on the CTA shows that four phases, with constituent teacher skills, are important for providing DI: preparing a lesson series, preparing a lesson, enacting a lesson, and evaluating a lesson. When preparing a lesson series, the first phase of providing DI, teachers analyze student characteristics and student performance to check the achievement levels and learning preferences of their students. They also make a plan for the lesson series, in which they determine what curriculum and homework will be part of each lesson. In the second phase, teachers prepare a single lesson within the lesson series. They map out the students’ starting point and determine the lesson’s goal (i.e., what students should have learned by the end of the lesson). They also determine the instruction, deciding what explanations and assignments they are going to use to achieve the lesson goal, and in their lesson plan, they determine when they will explain what part of the theory or when to discuss which assignment. In the third phase, the teacher teaches the lesson. They start by introducing the lesson, for example, by discussing the subject and goal of the lesson. Next, teachers give instruction aligned with the students’ achievement levels and stimulate students’ self-regulation by giving them responsibility for their learning process while coaching them. Teachers end this phase by wrapping up the lesson. They reflect on the lesson and might discuss a difficult assignment. During the entire lesson, teachers monitor the students’ progress, as well as who has or has not yet achieved what lesson goals. The final phase is evaluating the lesson. Teachers perform this for the short term (i.e., how this lesson went and whether they need to explain more in the next lesson) and for the long term (i.e., reflecting on what they could do differently next school year). The teacher skills required are strongly related to one another; for example, teachers can only provide aligned instruction (enacting a lesson) if they have analyzed student performance (preparing a lesson series).

Along with the required skills for DI, the CTA (Meutstege et al. [24]) also showed that three types of knowledge are necessary for providing DI in secondary education: knowledge about students, didactical–pedagogical knowledge and subject-matter knowledge. Teachers first gather knowledge about the students (e.g., their achievement level, motivation, and how well they work independently) when conducting continuous monitoring during the lesson. Without knowing their students’ learning needs, teachers cannot provide well-suited instruction. Teachers use the second type of knowledge, general didactical–pedagogical knowledge, that is, knowledge about how students learn and what instructional activities can help them, when deciding how to offer students appropriate support, for example, how to help a student who is insecure and how to organize and manage the classroom. Lastly, teachers need to have knowledge of the subject matter they teach. This includes knowing the curriculum and the learning path (a determined route from intermediate goals and content towards a final learning goal, used to arrange learning resources [26]), which can enable teachers to take a step back for low-performing students or a step forward for high-performing students. If teachers are to offer instruction that is well-suited to the needs of their students, they need to know what adaptations they can make to the instruction.

Finally, the CTA revealed that certain factors make providing DI in secondary education more or less complex [24]. These factors can be divided into four categories: class characteristics, school organizational characteristics, information about student achievement, and support from the curriculum. The characteristics of a class include the number of students, variation in student levels, class attitude (e.g., can the students work well independently), and classroom ambience (e.g., do students feel comfortable asking questions). For example, having a class with fewer students makes providing DI easier, as it is easier to know the students. Lesson duration, preparation time, time spent with students, number of activities in yearly planning, and physical space in classrooms make up the school organization category. For example, DI is easier for teachers if they have more preparation time and if there is more physical space to move tables around and to sit apart with a group of students. The third category, information about student achievement, refers to the richness and quality of the information about students’ progress that is available, and support from the curriculum refers to the fact that implementing DI is easier if the curriculum provides assignments for students of different achievement levels.

In conclusion, the CTA by Meutstege et al. [24] showed that providing DI in secondary education is a very comprehensive and complex task, for which teachers need to master numerous different skills with respect to lesson (series) preparation, lesson enactment, and lesson evaluation, as well as various types of knowledge, and there are several factors influencing the complexity of providing DI.

4. A Whole-Task Approach to the Design of the TPD Program: The 4C/ID Model

The complexity of providing DI in secondary education makes the design of a suitable TPD intervention for it also complex. A TPD intervention that teaches the various skills required for DI separately might lead to a poor transfer of learning; such a fragmented approach does not teach the learners how to integrate the newly learned skills [27]. The CTA [24] showed that the teacher skills that DI requires are strongly interrelated (e.g., a teacher provides adapted instruction based on what they monitored, they monitor where students stand with regard to the lesson goal determined by them, and so on). The CTA also showed that several types of knowledge are very important for providing DI, which is in line with Smale-Jacobse et al. [1], who state that a focus on both content and pedagogical knowledge may facilitate an effective TPD intervention for DI. A TPD intervention should take the relationships between the required teacher skills and the integration of the required skills and knowledge into account, as “it is this integration that facilitates the transfer of learning” [28] (p. 460).

An approach that integrates the required skills and knowledge is the whole-task approach. With a whole-task approach, learners work on tasks that represent the whole-task domain in its entirety and focus on the relationships between the different components [29,30]. A whole-task approach, such as the four-component instructional design (4C/ID) model [23], is useful for designing comprehensive TPD for DI. The 4C/ID model is about authentic and whole-task experiences presented to professionals in professional development programs. These tasks integrate the knowledge, attitudes, and skills that are required for performing complex professional tasks, such as providing DI. The 4C/ID model has been used successfully in many areas, including the design of an information problem-solving course in higher education [31], evidence-based medicine training in medical education [32], and a programming course for primary education [33]. Costa et al. [28] conducted a meta-analysis of the effects on learner performance of using the 4C/ID model for the design of educational programs, including 12 studies in elementary education, high school, and higher education in the domains of computer science, physics, mathematics, engineering, and health. They showed that training programs based on the 4C/ID model had a high average effect on student performance (Cohen’ s d = 0.79).

With the 4C/ID model, training programs are built on the basis of four basic components: learning tasks, supportive information, procedural information, and part-task practice, which are depicted in Figure 2.

Learning tasks provide the foundation for the training blueprint and are shown as circles in Figure 2. The learning tasks offered should be authentic, whole-task experiences incorporating the lack of structure, multiple disciplines and collaboration that are also required in real life, as this supports transferability to professional situations [30]. There should be variability in the learning tasks, as tasks in the work field will also differ (as indicated by the different positions of the triangles within Figure 2). Learners should work on tasks that are ordered from easy to complex, as this prevents cognitive overload [30]. Tasks at the same complexity level are clustered together in a task class, in which both support (built into learning tasks) and guidance (given by the trainer) should start high and decrease over time. This is also called scaffolding and is shown by the decreased fill level of the circles within a task class from left to right in Figure 2.

Supportive information helps learners (in our case, teachers) to carry out the nonroutine aspects of learning tasks that require decision making, problem solving, and reasoning. It describes the organization of the task domain to the learner in mental models and helps learners to connect new information to their prior knowledge. This results in systematic approaches to problem solving (SAPs): a set of goals sequenced so that they lead the learner through the task in an optimal order for successfully completing the task. Supportive information is identical for all learning tasks at the same level of complexity, because these tasks make an appeal to the same knowledge base. In Figure 2, supportive information is indicated by the L-shaped figures below and to the left of the task classes.

Procedural information helps learners carry out the routine aspects of learning tasks; that is, aspects that are always performed in the same way. This information often has the form of a step-by-step guide that couples certain cognitive actions (e.g., IF–THEN relationships) and should be provided just in time, preferably when the learner performs this part of a learning task for the first time. Over time, procedural information should fade until it has become an automated schema for the learner. Procedural information is indicated in Figure 2 by the beam under the task classes with upwards-pointing arrows.

Part-task practice is used when learners are required to have a high level of automaticity of routine aspects before starting to work on learning tasks. It is important for learners to first be introduced to the routine aspect in the context of the whole task, as this provides an understanding of the part-task practice’s importance. Part-task practice is shown by the small circles in Figure 2.

Combining these four components from the 4C/ID model can result in a professional development plan appropriate for training for a complex teacher task such as providing DI, as was shown by Frerejean et al. [34]. The next section describes how we used the 4C/ID model to translate the outcomes of the CTA of Meutstege et al. [24] into a TPD trajectory for DI in secondary education.

5. The Design of a TPD Program for DI in Secondary Education Using 4C/ID

5.1. The Design Team

This project is a cooperation between Stichting Carmelcollege (a Dutch school board) and the University of Twente, with the goal of designing, developing, and implementing a TPD trajectory for in-service teachers to help them become more proficient at providing DI. As the Dutch secondary school system is tracked (i.e., when starting secondary school at age 12, students are sorted into tracks based on their cognitive level), the TPD program should apply to teachers teaching in every track. It was also decided that the TPD program should be subject-specific; therefore, the TPD program focused on mathematics teachers. The design team consisted of a mathematics teacher with expertise in DI from Stichting Carmelcollege and a researcher (the first author) with expertise in DI from the University of Twente. The mathematics teacher was also the trainer for the TPD program.

5.2. Design Choices

The TPD program was intended for practicing teachers, who have ample practice opportunities in their own classroom, as Dutch secondary school teachers generally spend 45.2 h per week on their work (the two biggest categories are 14.6 h of teaching and 21.1 h of preparation/checking student work [35]). Hence, we decided to use workplace-based learning, incorporating all of the teachers’ mathematics classes as (opportunities for) learning tasks, which provides authenticity and automatically features the variability of real-life learning tasks (as the 14.6 h of teaching and 21.1 h of preparation/checking per week are all different).

The 4C/ID model states that the learning tasks should be ordered from simple to complex, while within a task class, support should start high and gradually become lower (i.e., scaffolding). In workplace-based learning, however, it is hard to influence the complexity of the learning tasks. For example, the CTA [24] showed that the number of students influences the complexity of DI, but it is not realistic to send half of the students home for the day to have their teachers practice their DI skills in an easier context. We solved this by applying emphasis manipulation [34], where the participating teachers’ attention is focused on a specific aspect of the learning task (e.g., monitoring) while still performing the whole task (i.e., providing differentiated instruction during a mathematics lesson). The chosen learning task aspects for the emphasis manipulation are the four phases of providing DI (lesson series preparation, lesson preparation, lesson enactment, and lesson evaluation) found by Meutstege et al. [24] and five principles [34] for providing DI that experts teachers use throughout the four phases: (1) to work in a goal-oriented way, (2) to continually monitor students’ progress towards the lesson goal, (3) to challenge all students, (4) to adapt the instruction to students’ needs, and (5) to stimulate students’ self-regulation.

A task class in the current TPD design is a cluster of learning tasks in which the same aspect (e.g., “preparing a lesson” or “monitoring”) is emphasized. Although the emphasized aspects might seem to stand alone, in practice, there is much overlap. For example, teachers’ preparation for monitoring their students’ progress towards the lesson goal is part of both “monitoring” and “lesson preparation”. For the TPD program, this means that it can be part of one of the task classes or both of them, depending on where it fits best.

We wanted to design a TPD program that provides tools and examples for teachers regarding how they can provide DI, not to tell teachers exactly how they have to do it. The quality of DI depends on the deliberate decisions teachers make, and we sought to provide them with the tools to make those decisions. Participating teachers are asked to record four of their lessons during the intervention and to share the videos with the coach and their peer teachers in the intervention group. On the one side, reviewing their own classroom videos was expected to promote reflection on their own DI practices; on the other side, sharing and discussing videos was expected to facilitate communication and collaboration between participating teachers [36]. Watching peer teacher videos can empower teachers and can make them feel less isolated in their classrooms, and it can also provide inspiration for their own practice, as all participating teachers were experimenting with implementing differentiated instruction.

Teachers differed in their teaching preferences, their teaching experience, and their level of DI before starting the TPD program. To take these differences into account, teachers will be asked to set personal learning goals on which to focus throughout the TPD program.

The CTA [24] showed that there are no routine aspects in providing DI. Although some aspects are always present (such as “provide adapted instruction”), the way they are executed is not always the same (e.g., extended instruction might not always be necessary for low-performing students). As there are no routine aspects, there is no procedural information or part-task practice for routine aspects in this TPD program.

The TPD trajectory starts off with a longer session about the task of providing DI as a whole, in order to emphasize what all the aspects of the whole task are and how they are related to one another. During this first session, it is also stressed that providing DI is about the deliberate decisions teachers make, as well as that throughout the TPD, participating teachers will be provided with the tools to make those decisions. All other sessions are focused on one or a few aspects of the whole task, as can be seen in Figure 3.

Two types of task classes were designed. One type is task classes that emphasize aspects of DI that are performed outside of the classroom, which applies to the task classes that are part of sessions 2 (prepare lesson series), 3 (prepare lesson), and 4 (evaluate lesson). The participating teachers practice these during the sessions, for example, by preparing their own upcoming mathematics lesson. The other type is task classes that emphasize DI aspects that are performed inside the classroom, which applies to sessions 4 to 8 (which emphasize introducing and wrapping up lesson, monitoring and providing adapted instruction, and stimulating self-regulation). As participating teachers practice these aspects in class, incorporating scaffolding is challenging (i.e., the trainer would interrupt a lesson too much if they provided in-class support). For these task classes, the trainer observes all participating teachers while they are teaching during the observation rounds (during which the emphasis is still on the aspect of the previous session). During the following session, the participants together with the trainer reflect on the observation round and discuss their experiences. In the rest of this section, an example is given for each of these two types of task classes.

5.2.1. Example of a Task Class Occurring Outside the Classroom (Type 1)

The first type of task class emphasizes aspects of DI that take place outside the classroom. This includes preparing a lesson series, preparing a lesson, and evaluating a lesson. Figure 4 shows the blueprint for the aspect “preparing a lesson series” as an example of what task class type 1 looks like. For the blueprints for the other task classes in this category, see Appendix A.

The first column shows when something takes place, where the S stands for session and OP stands for own practice. It can be seen that all of the supportive information and most of the learning tasks occur during session 2, except for the final learning task, which takes place in the teachers’ own practice.

The second column shows the degree of scaffolding by indicating how much support the participating teachers receive for carrying out a learning task. As can be seen, participants first receive a high level of support (participants watch and discuss a modeling example), and thereafter, this decreases to a lower level of support (planning your own lesson series with these 5 steps while a trainer is present) and then to no support at all (later on, in their own practice, participating teachers prepare another lesson series on their own learning task OP2).

The third column describes whether an element is supportive information (shaded) or a learning task (unshaded) and gives a short description of the element. The elements related to supportive information are shaded, while the learning tasks are not. The task class starts off with supportive information showing participants how this aspect of the task (in this example, to prepare a lesson series) is organized and what steps they can take to execute this aspect themselves (i.e., a SAP). The examples (modeling examples and case studies) are authentic, and the participants also design multiple lesson series of their own. Hence, there is much variability in the learning tasks. For task class 2 (lesson series preparation), there is not much practicing during their own practice, as preparing a lesson series is something that happens relatively infrequently. For task class 3 (preparing a lesson), however, every lesson is a learning task, and thus participants have ongoing opportunities to practice their newly learned skills.

5.2.2. Example of a Task Class Occurring during Actual Teaching (Type 2)

The second type of task class emphasizes aspects of DI that are to be practiced in the classroom: introducing and wrapping up the lesson, monitoring student performance, providing adapted instruction, and stimulating student self-regulation. These task classes have more or less the same structure. The blueprint for task class 6 (emphasis on monitoring) is provided as an illustration in Figure 5; the blueprints for the other task classes of this type can be found in Appendix A.

The first column shows when something takes place; just as in Figure 4, the S stands for session and OP stands for own practice. The entries in this column show that the task class starts in session 5, followed by an observation round (OR), reflection in session 6, and finally, it happens in their own practice (OP).

The second column again shows the amount of support that participating teachers receive. Although the support can be scaffolded for task class type 1, the fact that teachers are practicing in their classroom complicates scaffolding support for task class type 2. To ensure that participants still receive enough support in task class type 2, they receive more support before and after practicing in class.

The third column elaborates on what the element entails and whether it is supportive information (shaded) or a learning task (unshaded). The task class starts off with supportive information, enabling a shared image of what is meant by “monitoring”. For the learning tasks in session 5, participating teachers study high-quality examples of monitoring, reflect on how they already monitor students, and prepare how they will monitor during an upcoming lesson (i.e., the task class “prepare a lesson” continues and is practiced again during this task class). Teacher support stays high to medium. Next is the observation round, where over the span of 3 weeks the trainer visits each of the participating teachers in their classroom to observe them while they practice “monitoring” during teaching and to provide cognitive feedback, which is feedback focused on fostering reflection for the learner [30]. In the following session, participating teachers and the trainer reflect together on the observation round by watching video clips that show how the teachers practiced the emphasized aspect. Finally, the participants keep practicing the newly learned skills in their in-class lessons for the rest of the TPD program.

The final result is a TPD trajectory that takes place over a period of 5 months so that teachers have sufficient time to practice the complex skill of DI.

6. Evaluation of the Pilot Training

From January to June 2022, four secondary mathematics teachers participated in a pilot of the TPD program. In this section, we describe a first exploration of how the participants experienced the TPD program, the results of which can be used for improving the program.

6.1. Method

6.1.1. Data Collection

To analyze what parts of the TPD intervention could be further improved, the pilot was used for beta testing. Beta testing is the study of the functionality of an educational intervention and how it interacts with its context [37]. The goal was to analyze what aspects of the intervention foster or hinder success. For data gathering during a pilot, McKenney and Reeves [37] mention interviews as the most useful method.

For the interviews in this study, the exit cards that the participants completed after some of the sessions (a few short, reflective questions such as “What was an eye opener for you today?” or “Would you recommend this TPD program to colleagues?”) and the storyline assignment (a graph where the participating teachers indicated how much they had learned during each session) they completed during the final session were used as input. This input was supplemented with more general questions, such as “What did you think about the sessions?” or “How could the TPD program have suited you better?”

6.1.2. Procedure

The teachers received an email during the final stage of the intervention with the question of whether they wanted to participate in an interview, to which all four answered positively. The interviews took place within two weeks after the last session of the TPD program. A researcher visited the participant teachers at their schools and interviewed them in their own classrooms. The interviews were audio-recorded. While all participating teachers provided active written informed consent for the gathering of data throughout the TPD intervention (e.g., the exit cards and storyline), they were also specifically asked if they agreed to take part in this interview, which was voluntarily, and there was no requirement for participation in the TPD intervention. This study was approved by the BMS Ethical Review Board of the University of Twente (approval no. 211337).

6.1.3. Data Analysis

First, the interview recordings were transcribed (edited transcription, not fully verbatim) to enable further analysis. The transcripts were then coded following an inductive approach [38]. Codes were clustered to identify common themes in respondents’ answers. Examples of codes are “subject specific” and “cognitive feedback”. Following a deductive approach based on the two types of task classes in the TPD (type 1: task aspects performed outside the classroom; and type 2: task aspects performed during actual teaching) and the teachers’ experiences with the TPD intervention in general, the coded segments were clustered in those three categories to enable us to answer the research questions at hand. The interviews were conducted in Dutch; hence, the questions and quotes in the current paper are translated.

6.2. Results

The four participating teachers’ opinions regarding their experience in the TPD program are organized below into three sections. The first section is about participants’ general experiences with the TPD program, and in the second and third sections, teachers’ experiences with the two types of task classes are discussed.

6.2.1. How Did the Participating Teachers Experience the TPD Program in General?

In general, all participating teachers were positive about the TPD trajectory and would all recommend participation in the TPD program to colleagues. Frank said, “Definitely! It is not only about awareness, but also about providing the techniques/tools to get things done”. John said, “I think this is very useful for anybody, also for colleagues that teach other subjects”. Paul summed up the TPD program as follows: “This is all working together on the most important part of our job: interaction with students”.

What made the TPD experience good for the participants was that the program was well-suited to their practice. Frank said, “[What makes this attractive] is that you immediately apply it, you can immediately see how it works, the monitoring, you reflect on it”.

They were also pleased that the sessions were held in small groups. Although Frank initially was afraid that the group would be too small, his experience was that everyone had the opportunity to participate in discussions. He said, “By tackling one aspect together and working on it individually you get experience with it and then subsequently discuss it. That gives a lot of insight into the possibilities, what you could have done differently”. Paul found that “if the group had been bigger there is not a good chance that everyone would be able to show their video recording, and that was such a strong point now”.

Another positive point mentioned was that all participating teachers teach the same subject, as this made all the experiences and examples of colleagues more relatable. Paul said, “A remark that someone else made is that it is very nice that you participate in the training together with math teachers. Or maybe mathematics, physics, or chemistry, as the content is similar. Because grammar or literature, those examples are of no use to me”. Subject specificity included the fact that the trainer was also a mathematics teacher. Paul said, “[The trainer] is just a mathematics colleague and if you look at a lesson together with a mathematics colleague, you always improve as a teacher. We should do that way more often”.

The participating teachers were not unanimous in their opinions about whether the cost–benefit balance of this way of providing DI was good or not. Some found that it cost too much time. Kevin mentioned, “You cannot take an hour for preparing a lesson every time. That is not possible. The way I prepare lessons now takes me three hours. … For an observation round I will do it once, but that is not feasible”. Paul’s first answer was similar: “If you want to do it as intensively as we did during the training, the preparations will take longer than the lesson itself”. Later in the interview, however, he said, “I noticed that I secretly made use of it often and it definitely paid itself back. … I found that the time spent and what you got back from it were well balanced”. John went even further than that and found the cost–benefit balance to be very good: “I think the course has been very valuable because I have way more peace during the lessons itself. I notice that if I have to teach six lessons consecutively, I prefer to provide six of these [lessons wherein I provide DI] lessons than six normal ones. It really makes a difference for me. I think they get more focused attention now”.

6.2.2. Experiences with Task Class 1: Task Aspects Performed Outside the Classroom

The participating teachers found that in the sessions highlighting the task aspects that are performed outside the classroom, they benefitted most from the tools and examples provided, such as the modeling examples. John said, “I think I learned the most from the video clips and discussing them afterwards”. He found that by viewing concrete teaching practices and talking about those practices, he became more conscious of those things. Another benefit of the modeling examples according to Frank was that “You immediately see how students react and whether it works or not”.

Despite the fact that they found that the tools and examples they received were good, some of the participating teachers still found it hard to think of ways they could provide DI in their own lessons. Kevin said, “I find it very hard to do that on my own. … I just find it hard to find inspiration or to do something creative”. John especially would have liked more examples of how to provide adapted instruction for high-performing students: “For the high-performing students the standard curriculum is sometimes not sufficient and then you need extra stuff, which is sometimes hard to find”. Kevin found that he spent way too much time preparing a lesson. As a suggestion, he said that “Maybe we should be more active about everybody’s lessons during the sessions. Like, what are you actually going to do during the upcoming observation”.

6.2.3. Experience Task Class 2: Task Aspects Performed during Actual Teaching

All participating teachers stated that they found the lesson observations and cognitive feedback one of the best parts of the TPD program.

John found that the lessons observations forced him to actually get started with differentiation in the classroom: “Otherwise I am inclined to think ‘Oh, I can also start at a later date’. Yeah, I think I need [the observations]”. Both Kevin and John mentioned that they found that the observations ensured that they made more deliberate decisions during lesson preparation.

They appreciated the personal cognitive feedback a lot. “During the lesson you cannot teach and reflect at the same time to see if it worked as intended and if you taught the students what you wanted to. [The trainer] does show that, which I found very useful”, said Paul. They felt, however, that the trainer could have been a bit more critical at times. Frank said, “I think she could be more critical toward me and raise the bar a bit. … You can ask critical questions and I think I could have been challenged a bit more”. Some found it a pity that the feedback they received after the observation led to them being extra active about the highlighted aspect only after the lesson observation round, when the focus of the TPD was already on a new aspect. For example, this happened in task class 6, where continual monitoring was highlighted. Paul said, “I came up with how I would monitor and what I would do specifically only afterwards with the feedback and viewing my colleagues’ video clips to see what I could have done better”.

The participants found viewing each other’s video clips in the reflection sessions a good source of inspiration. John mentioned he thinks he learned most from watching the video clips and discussing them afterwards. Frank found sharing the video clips inspiring, “Because you get ideas, like someone having a certain set up of their students’ tables and you think: ‘Hey, I want to try that out sometime’”. He also mentioned, however, that the discussion afterwards could be a little more in-depth: “You can say ‘I’ll highlight adapting the instruction,’ but that is still abstract. What options did you think about and what ideas did you have and why did you choose something? Those are important questions. And you don’t see those when you view a lesson in hindsight. Then you see the result”.

7. Conclusions and Discussion

In the current paper, the design of a TPD intervention for secondary school mathematics teachers to improve their skills for providing differentiated instruction was described in detail. This TPD is based on deliberately chosen intervention content (the results of a cognitive task analysis of DI in secondary education [24]) and a deliberately chosen intervention method (the 4C/ID model [23]). The pilot of this TPD program lasted 5 months, in which the four participating teachers studied modeling examples from mathematics teachers, prepared how they would provide DI in upcoming lessons and lesson series during sessions, received personal cognitive feedback based on in-class observations, and reflected together on their progress. They practiced their newly learned DI skills in their own mathematics classes so that they could immediately experience what worked for them and what did not, as well as how their students experienced their adaptive teaching. All teachers were positive about their participation in the TPD program.

This paper is one of the first to describe the theory-based design and content of a TPD intervention for secondary education aimed at improving DI skills in such detail. This fills a gap in the existing knowledge about what TPD interventions actually consist of and why they are designed in a certain way. If other researchers would also describe their design approach and TPD intervention content and method and evaluate the results of the interventions systematically, then conclusions could be drawn about what TPD characteristics work and what do not. We piloted an evaluation of the impact of the TPD program for teachers and think that the next step should be a larger-scale evaluation of TPD program effects on teachers and students. Finally, detailed descriptions of TPD programs make it easier to replicate (parts of) interventions.

7.1. Limitations

A limitation of the current study is that only four teachers participated in the pilot. This means that the conclusions that we can draw are limited. We also do not yet know whether or not the participating teachers improved their DI skills as a result of participating in the TPD program. Guskey [39] distinguished between five levels of evaluation of impact of professional learning interventions: (1) participants’ reactions, (2) participants’ learning, (3) organizational support and change, (4) participants’ use of knowledge and skills, and (5) student learning outcomes. In the current paper, we presented the results of an intervention on level 1: the reactions of the participants. In future studies, the TPD program will be evaluated on the other levels.

Another limitation is that this TPD intervention was specifically designed for mathematics teachers. This was performed deliberately, as this makes the examples and experiences of participating colleagues more relatable. As subject specificity can have a positive influence on the outcomes of the TPD [11] and was mentioned by the participants as a beneficial aspect of the TPD, a suggestion for future research is to design and implement similar TPD programs for other subjects, such as the languages and social sciences.

7.2. Recommendations for Future TPD Interventions

Several recommendations for future TPD interventions have been identified based on the current study. The first is to connect the TPD intervention with a subject domain. In our TPD program, all participating teachers, as well as the trainer, were mathematics teachers, just as the (modeling) examples used were about mathematics (lessons). The participants found that this ensured that they could relate to the provided examples and to the video clips of their fellow participants. This could explain why Kahmann et al. [11] found that “subject specificity” can have a positive effect on TPD outcomes.

We also recommend TPD facilitators to encourage their participating teachers to immediately apply their newly learned skills in practice, while the trainer visits them in class for observations and provides them with cognitive feedback afterwards. The participants found the observations helpful for actually getting started with providing DI and prepared their lessons more deliberately. The cognitive feedback afterwards helped them reflect on their own lessons.

The third recommendation is related to the complexity of providing DI and therefore the complexity of learning to provide DI. At the end of the TPD program, even though we strived to reduce cognitive load through scaffolding and emphasis manipulation, some of the participants still found providing DI very complex. We therefore recommend designers of future TPD interventions with a focus on DI or other complex teacher skills to (1) make smaller steps in their emphasis manipulation (e.g., divide the preparation of a lesson series into multiple steps), (2) provide more support for lesson preparation for certain aspects of type 2 task classes, and (3) design a TPD intervention that lasts longer than 5 months in order to offer more and longer support.

Another recommendation is to promote the sharing of ideas and examples. The participants mentioned that viewing video clips of their fellow participants provided them with new ideas. This is in line with the findings of Borko et al. [40] that teachers found inspiration for new pedagogical strategies when watching video clips of colleagues in action. Some participants stated that they spent much time thinking about ideas for their lessons, for example, about extra challenging assignments for high-performing students. If participants in an iteration (one round of participants) of the training, or even participants in earlier iterations, had a good space to share their ideas and examples, teachers would not have to reinvent the wheel as much, which might reduce their time spent on lesson preparation.

The final recommendation is to take teachers’ professional experience into account in TPD design. The TPD intervention described in this paper was designed for teachers with varying levels of professional experience. Ideally, scaffolding would gradually diminish if possible so that teachers are always working in their zone of proximal development. In this way, the TPD intervention can differentiate better between teachers’ learning needs.

In the current paper, we presented the design of a TPD intervention for which the content and design approach were deliberately chosen. The content was based on work by Meutstege et al. [24], who conducted a CTA that resulted in a practice-based, feasible overview of how experts provide DI. The CTA outcomes were used as the base for applying the design approach: the 4C/ID model [23], which is a validated whole-task approach to TPD design. As far as we know, we are one of the first to give such a thorough description of the intervention content, method, and design approach for a TPD program aimed at improving the DI skills of secondary school teachers. The next step is to evaluate this TPD intervention on a larger scale with a randomized controlled research design. We urge other designers and researchers to also provide thorough descriptions of the TPD interventions they implement and/or evaluate. In this way, our knowledge regarding effective TPD for DI can grow. Hopefully, in the future, we will be able to improve teachers’ DI skills and, in this way, ensure that as many teachers and students as possible will profit from the benefits of DI.