Understanding Assessment Processes for Communicative Competence through an Analysis of Teachers’ Reported Practice

Round 1

Reviewer 1 Report

Review of Manuscript ID: education-770504

Understanding Assessment Processes for Communicative Competence through an Analysis of Teachers’ Reported Practice: A Case Study of Spanish Primary Education

This manuscript reports on case studies to determine how communicative competence is taught and assessed in four selected schools in the south of Spain. The introduction informs the reader about the importance of developing good communicative competence, especially at the elementary level, and how it is related to the Common European Framework of Reference for Languages. However, the introduction does not cover several issues. One of them is a vague reference to the present state of elementary education in Spain. This can be seen in lines 42-47 with the sentence starting as “In the case of Spain, the inclusion of communicative competence in the school curriculum has resulted in some imbalances…” The reader has no idea what the imbalances are or their impact on the investigated topic. Another missing part is the Review of Literature. Perhaps in this review, the authors could have established the importance that some concepts presented later in the manuscript have to communicative competence and they could have defined them, such as textual macrostructure and genre (line 150), meta-discursive background (line 152), textual microstructure (page 10), etc.

The Research Design seems to be adequate to conduct this investigation, although the author(s) did not explain why these precise schools were selected, as they seem fairly similar. For instance, schools 1 and 2 are low performing; schools 2 and 3 are situated in low to medium SES neighborhoods; and schools 3 and 4 are both private and are situated in middle-income neighborhoods. In terms of Methods, their description is confusing. An example is Table 2, Calculation of Kappa (line 121) where the reader does not have information about the meaning of the columns next to the Kappa coefficient nor their meaning. Also missing is how the coding system was applied when the results were presented. In terms of Results, it was good that the authors discussed each research question separately. As discussed above, this section was also confusing. Table 4, Frequencies and total percentages of the categories by school, has some numbers that are shaded but they are not explained why they were presented that way. An explanation, in narrative form, of the table will help the reader understand its significance, especially about how the coding system was applied to identify the different cases.

The Conclusions still present new information that makes the reader question the content of the manuscript. Specifically, on the bottom of page 11, the following is stated: “In Spain, the state and regional standards prescribe the implementation of key competences, but curricular materials facilitating this process and serving as a reference for teachers have not been developed, thus increasing uncertainty with respect to their implementation, and especially their assessment.” After discussing how much teachers focus on mostly one of the elements of communicative competence (i.e., writing), the reader learns that the lack of materials that have had a big contribution to how teachers teach and assess.  This reviewer is uncertain about how the information presented supports the stated conclusion, “In conclusion, this research reiterates the need to reconsider the curricula and their content, and the methodologies and assessment procedures to develop communicative competence [71]” (page 11).

Author Response

Point 1

This manuscript reports on case studies to determine how communicative competence is taught and assessed in four selected schools in the south of Spain. The introduction informs the reader about the importance of developing good communicative competence, especially at the elementary level, and how it is related to the Common European Framework of Reference for Languages.

However, the introduction does not cover several issues. One of them is a vague reference to the present state of elementary education in Spain. This can be seen in lines 42-47 with the sentence starting as “In the case of Spain, the inclusion of communicative competence in the school curriculum has resulted in some imbalances…” The reader has no idea what the imbalances are or their impact on the investigated topic.

Response 1:

The authors have modified the text incorporating the following clarifications suggested by this reviewer:

“A detailed analysis of the process followed by those European countries integrating the competences into their curricula was carried out by Arjomand et al. [12], Gordon et al. [13] and Halász and Michel [14]. In the case of Spain, the inclusion of communicative competence in the school curriculum has resulted in some imbalances between either, the curriculum design and the teaching and learning processes carried out by primary education teachers. These imbalances have their origin in the way in which the competences were introduced into the curriculum [15,16,17,18]. The key competences have not been incorporated as the central an integrative curricular axis, but they have been just added to the existing curriculum. In this way, the function of the competences have not been to reformulate the design and development of teaching and learning processes, but to accomplish with the European guidelines. As a consequence of this superficial fit, the subject areas and the assessment standards continue focused on the contents of school subjects and not on the competences [19,20,21,22,23,24]. The last Spanish educational law in 2013 [2] even reduced the role of key competences compared to the legislation of 2006 [1], conceiving them more as capacities than as elements applicable to all disciplines [25].”

“Along with the scarce policies to really integrate the competences in the Spanish education system, the teachers’ training to develop and assess them becomes an added difficulty. Teachers (…)”.

Point 2

Another missing part is the Review of Literature. Perhaps in this review, the authors could have established the importance that some concepts presented later in the manuscript have to communicative competence and they could have defined them, such as textual macrostructure and genre (line 150), meta-discursive background (line 152), textual microstructure (page 10), etc.

Response 2:

The authors have modified the text incorporating the following clarifications suggested by this reviewer:

“The communicative competence may be understood as the development of listening, reading, speaking and writing through the interaction of different components, such as the linguistic (lexical, grammatical, semantic and phonological dimensions), the pragmatic-discursive (sociolinguistic and pragmatic suitability of the discourse to the different situation contexts), the socio-cultural (the discursive genre chosen in each communicative situation) and the strategic (integration of non-linguistic and multimodal components along with linguistic ones) [1v]. The development of communicative competence requires the acquisition of a meta-discursive awareness in children. According to the schemes proposed by [2v] and [3v] shown in Table 1, this metacognitive awareness described by [4v] and [5v] enables the further development and accurate use of different discourse levels by speakers, such as the textual microstructure [6v], the textual macrostructure [7v] and the genre [8v,9v].

Table 1.

Relations between genre, textual structure and language levels

Textual structure	Language levels	Text descriptors
A. Genres	A. Text	Basic models of text building according to their field, tenor (to whom it is adressed) and mode (medium of release), configured by use through History. They have specific pragmatic, structural and linguistic conventions.
B. Macroestructures	B.1. Theme	The topic and intentionality of the autor summarised by the main idea of the discourse.
	B.2. Structures and etages	The different parts of the text are interconnected, conforming a thematic progression. It allows the theme of the text to expand in subthemes, and the different narrative, descriptive or expositive sequences to progress. According to its genre, the text displays a structure enabling it to be identified as belonging to a particular genre.
	B.3. Cohesion	The linguistic elements facilitating the global and lineal coherence of the oral and written text to be displayed. The linguistic tools that make it possible are: a.         References. b.         Substitutions. c.         Ellipsis. d.         Theme and rheme. e.         Conjunction.
C. Microestructures	C.1. Clauses and sentences	This level addresses the accuracy of the syntactic construction and its relation with the genre or textual sequence it belongs to.
	C.2. Words	Lexical level addresses the words choice from a lexical-semantic perspective (paradigmatic axis).

Point 3

The Research Design seems to be adequate to conduct this investigation, although the author(s) did not explain why these precise schools were selected, as they seem fairly similar. For instance, schools 1 and 2 are low performing; schools 2 and 3 are situated in low to medium SES neighborhoods; and schools 3 and 4 are both private and are situated in middle-income neighborhoods.

Response 3

The authors have modified the text by incorporating this paragraph at the beginning of the section Method, following as suggested by this reviewer:

“Our article addresses the research as a multiple-case study [49], with a descriptive and exploratory aim. To this end, a thematic analysis was carried out [50,51] thereby determining a series of choices concerning the research’s design and analysis. The first of these choices is analysis from the critical realism perspective, as it adopts different valid perspectives concerning the same event [52].

Besides, the authors have modified the text incorporating the following clarifications suggested by this reviewer:

“The schools were deliberately chosen. Once they were informed about the aims of the research, they showed special interest in taking part in it. Apart from these four centres, another two schools were initially reached, although they finally refused to participate due to technical problems –the required internet connection- and their lack of time to pursue the teacher training period included in the research.”

Point 4

In terms of Methods, their description is confusing. An example is Table 2, Calculation of Kappa (line 121) where the reader does not have information about the meaning of the columns next to the Kappa coefficient nor their meaning.

Also missing is how the coding system was applied when the results were presented.

Response 4

The authors have modified the text incorporating the following clarifications suggested by this reviewer:

“The categories and their indicators were numbered in order to speed the coding process, and facilitate the subsequent data entry in SPSS. Each researcher coded the sentences assigning a number according to the identified indicator  In the data matrix generated for the analyses there were three columns –one for each researcher- and as many rows as codings. Thus, each row captured the numbers (indicators) assigned by each researcher to every single coding.”

The authors have incorporated the following clarifications after Table 3:

K = Kappa value; ASE = asymptotic standard error; Z = standardized values; P-valor = significance 2-tailed

In addition, "(K > .94)" is added in the previous paragraph to Table 3 to clarify the meaning of the data indicated in that Table.

Point 5

In terms of Results, it was good that the authors discussed each research question separately.

As discussed above, this section was also confusing. Table 4, Frequencies and total percentages of the categories by school, has some numbers that are shaded but they are not explained why they were presented that way. An explanation, in narrative form, of the table will help the reader understand its significance, especially about how the coding system was applied to identify the different cases.

Response 5

The authors have modified the text incorporating the following clarifications suggested by this reviewer:

“Table 5 shows all the condings performed, both for wach code and for each school (S. 1, S. 2, S. 3 y S. 4). The percentages highlighted in grey are the most remarkable data for our research, taking every school into account.  The interpretation of these results is unpacked below, giving response to the research questions of our study”.

Point 6

The Conclusions still present new information that makes the reader question the content of the manuscript. Specifically, on the bottom of page 11, the following is stated: “In Spain, the state and regional standards prescribe the implementation of key competences, but curricular materials facilitating this process and serving as a reference for teachers have not been developed, thus increasing uncertainty with respect to their implementation, and especially their assessment.” After discussing how much teachers focus on mostly one of the elements of communicative competence (i.e., writing), the reader learns that the lack of materials that have had a big contribution to how teachers teach and assess.  This reviewer is uncertain about how the information presented supports the stated conclusion, “In conclusion, this research reiterates the need to reconsider the curricula and their content, and the methodologies and assessment procedures to develop communicative competence [71]” (page 11).

Response 6

The authors have modified the text incorporating the following clarifications suggested by this reviewer:

“In conclusion, this research confirms the results of previous studies [80] and delves into the inquiry of the communicative competence. More specifically, our contribution has focused on the nature and types of the tasks undertaken in the classrooms, and on their assessment by the teachers.  Our results underpin the necessity of implementing, within the Spanish context [81], the advice from the European Union Council [3]. These advices are oriented towards three main lines of action: a. a wider diversity of approaches and learning environments in the school;

1. a specific support for teachers, oriented to the training in key competences;
2. an institutional plan aimed to train the teachers in efficient tecniques for the assessment of the key competences.

Our research results reveal the differences between the prescriptions of the Spanish curriculum in [1] and [2], and the tasks and assessment carried out by the primary education teachers nowadays. Such as [82], our study shows that teachers should receive appropriate training in assessment, enabling the development of the curriculum through the competences, and by doing so, transforming the teaching and learning process for students”

Author Response File: Author Response.docx

Reviewer 2 Report

Dear authors,
Congratulations for your manuscript!
My suggestions and comments are in the attached file.
Sincerely,
Rev.

 

 

Comments for author File: Comments.pdf

Author Response

Point 1: Title

That is a very long title.

It must be a short title.

I suggest, for example, to remove “a case study…

Response 1

The authors modify the paper by accepting this reviewer's suggestion. The title is "Understanding Assessment Processes for Communicative Competence through An Analysis of Teachers’ Reported Practice".

Point 2: Abstract

Remove.

you cannot generalize.

Response 2

The authors modify the paper by accepting this reviewer's suggestion

Point 3: Keywords

I have numerous reservations with this keywords.

I think have understood this choice. But, in fact, probably it might be better  introduced the concepts in the "Introduction", for example.

Response 3

The authors modify the keywords to adapt them to their theoretical framework:

Primary education; communicative competence; assessment literacy; assessment for learning

Point 4: Method

remove to line 82.

It should be before the subsection 2.1

Response 4

The authors modify the paper by accepting this reviewer's suggestion.

Point 5: Findings and Discussion

Attention to relation to how as written the references ( basis/support for your study ) to support your findings. See some my suggests in section "conclusion"

Response 5

The authors have modified numerous references in the text in order to improve their relationship with the arguments provided in the Findings and Discussion.

Point 6

I suggest:

study such as [18]

Response 6

The authors modify the text following the indications of the reviewer and rewriting it in the following way:

“This study such as [18] shows how assessment is carried out on the content and not on the dimensions of communicative competence.”

Point 7

I suggest:

This research corroborates previous studies [71]…

Response 7

The authors modify the text following the indications of the reviewer and rewriting it in the following way:

“In conclusion, this research corroborates previous studies [71] and reiterates the need to reconsider the curricula and their content, and the methodologies and assessment procedures to develop communicative competence.”

Point 8

I suggest:

such as [73]

Response 8

The authors modify the text following the indications of the reviewer and rewriting it in the following way:

“Such as [82], our study shows that teachers should receive the appropriate training in assessment, enabling the development of the curriculum through the competences, and by doing so, transforming the teaching and learning process for students.”

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The author(s) modifications greatly enhanced the readability of the manuscript. There are some misspellings and grammatical issues that the editor will probably change.

My only disagreement with the author(s) is that they name only the number of the reference instead of the authors of the reference but the practice is not systematic. That is, sometimes the numbers are mentioned, as in

"between the prescriptions of the Spanish curriculum in [1] and [2], and the tasks and assessment carried out"

and other times the authors' names and the reference number are provided, as in,

"Moya and Luengo [25]"

This reviewer believes that last names of authors should always be listed. 

Comments for author File: Comments.pdf

Author Response

The authors have reviewed the text taking care of all the reviewer's recommendations. The way in which the references are incorporated into the text has been normalized. The answers to each of the comments have been inserted in the PDF file attached to this message.

Author Response File: Author Response.pdf