Next Article in Journal
Integral Representation for the Solutions of Autonomous Linear Neutral Fractional Systems with Distributed Delay
Next Article in Special Issue
Collaborative Learning Based on Harry Potter for Learning Geometric Figures in the Subject of Mathematics
Previous Article in Journal
Delay Effect and Subadditivity. Proposal of a New Discount Function: The Asymmetric Exponential Discounting
Previous Article in Special Issue
Fractions and Pythagorean Tuning—An Interdisciplinary Study in Secondary Education
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Mathematics
Volume 8
Issue 3
10.3390/math8030368
mathematics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Secondary Education Students’ Beliefs about Mathematics and Their Repercussions on Motivation

by
Vanesa Rojo Robas
1,
José María Madariaga
2 and
José Domingo Villarroel
3,*
1
Department of Didactic and School Organization, Faculty of Education in Bilbao, University of the Basque Country (UPV/EHU), Barrio Sarriena, s/n, 48940 Leioa, Spain
2
Department of Developmental and Educational Psychology, Faculty of Education in Bilbao, University of the Basque Country (UPV/EHU), Barrio Sarriena, s/n, 48940 Leioa, Spain
3
Department of Didactic of Mathematics and Experimental Sciences, Faculty of Education in Bilbao, University of the Basque Country (UPV/EHU), Barrio Sarriena, s/n, 48940 Leioa, Spain
*
Author to whom correspondence should be addressed.
Mathematics 2020, 8(3), 368; https://doi.org/10.3390/math8030368
Submission received: 11 January 2020 / Revised: 13 February 2020 / Accepted: 3 March 2020 / Published: 6 March 2020
(This article belongs to the Special Issue Mathematical Modeling and Simulation in Science and Engineering Education)
Versions Notes

Abstract

:
This research analyses the role of the affective dimension in learning mathematics during secondary education. In particular, the evolution of motivation and beliefs according to the students’ academic level, as well as the relationship between these two factors. The sample consisted of 202 students from all four years of secondary education, whose motivation and beliefs about mathematics and learning mathematics were evaluated. Beliefs were grouped into three categories: related to the specific classroom context, to mathematics, and to oneself as a learner of this subject. The results obtained indicate lower motivation and less favourable beliefs in 3rd year. From the beliefs analysed, those regarding the classroom context, and more specifically the teachers, have obtained the largest effect size.

1. Introduction

Knowledge acquisition requires a certain balance between what is rational or cognitive and what is affective-emotional [1,2,3]. However, our education system focused mainly on the development of the rational mind, identifying this with scientific knowledge [4].
Thus, mathematics has become a very important part of the education of our students and as a consequence, a highly demanding subject [5,6]. At the same time, several studies show that many students display negative attitudes towards mathematics throughout their academic life [7,8].
This aversion to learning mathematics and the resulting high index of academic failure in this subject could be motivated by many factors in the affective-emotional domain (feelings of insecurity, anxiety, etc.) as was found in studies carried out from the 1980s onwards [7,9,10,11,12], or maybe also because there is a social belief that mathematics is a difficult subject [13,14]. This aversion could also have to do with teachers’ attitudes or with the methodology used in teaching mathematics [15,16,17].
As a consequence, the affective domain of learning mathematics has become an important field of study. The different studies developed in this field can be grouped around three main variables: attitudes, beliefs, and motivation towards mathematics [9,10,12,13].
Some previous studies have focused on the role of motivation in learning mathematics on the one hand [18,19] and the role of beliefs on the other [20,21]. Regarding motivation, [18] studied the impact of motivation on the academic performance of mathematics students in Nigerian secondary schools, concluding that these students differ significantly in their performance based on their motivation. The results reveal that highly motivated students perform better academically than less motivated students. Different studies support the idea that self-efficacy is associated with motivational orientation towards mathematics as well as with a better mathematical performance [22,23]. Regarding beliefs, [20] studied the influence of beliefs on academic performance in a sample of 1600 Spanish secondary school students. In addition to corroborating this idea, he found that epistemological beliefs not only directly influenced academic performance, but also contributed indirectly through their effects on student learning approaches.
In this paper, we will focus on the possible relationship between motivation and beliefs, and will try to establish possible relationships between different types of beliefs of secondary education students and their corresponding motivation, with the aim of assessing their resulting educational implications. In this sense, there is the antecedent of [24] who conducted a study to identify the key variables in the belief networks, under the assumption that knowing those variables helps to understand what motivates students.
However, what are we referring to when we talk about beliefs and motivation in learning mathematics?
Motivation is a fundamental variable to keep students engaged in the study of any subject and to lead them to an adequate and durable knowledge acquisition [25,26,27,28], and of course this also goes for mathematics, [15,18,19,29]. More specifically, we are referring to intrinsic motivation, which is the motivation that does not refer to an external reward, but rather whose motivating factors are inherent to the subject or the task being undertaken.
A students’ beliefs about mathematics are among the components of an individual’s implicit subjective knowledge of this subject and its teaching-learning [30,31]. Such beliefs are categorized according to their object as follows:
(a)
beliefs about oneself regarding learning and resolving mathematical problems
(b)
beliefs about mathematics, its teaching and learning
(c)
beliefs about the social context in which the educational experience of mathematics takes place.
In his preliminary studies [9,10], and other researchers later [21,32] found that all three types of individual students’ beliefs about mathematics and its teaching described above have a strong impact on the educational process, in the sense that such beliefs will condition students’ learning and the way in which they will use mathematics in the future. This is why these beliefs are considered the basis for generating knowledge and should be understood as a cognitive reference that conditions students’ affective aspects, and that might predispose them to act according to these beliefs [30,33,34].
Different research studies were developed in relation to each of the three types of beliefs described:
(a)
Beliefs about oneself as learner of mathematics have commonly been interpreted as the beliefs a student has about him/herself or about learning very specific tasks: solving routine and non-routine problems, fractions, proportions, algebra, geometry, and calculus. Students who self-perceive more capable in one subject or study area, are more prone to engage in learning-related tasks and get better marks. In this sense, [21] corroborated the relationship between students’ motivational beliefs and how they were reflected in the qualifications of teachers in science and mathematics. In the same way, the beliefs in the capacities to reach set goals influence the students’ motivation and effort put into learning. Thus, [29], after analysing the results obtained in a research of 336 secondary school students, revealed that positive self-concept and self-perception improve the students’ level of motivation, which increases their level of performance.
(b)
The beliefs students have about mathematics as a subject are related mainly with its importance and the perceived usefulness, and in turn with their success and level of participation in the mathematics class [35]. Mathematics is usually conceived as a complex discipline, difficult to understand, requiring great effort and the use of higher order cognitive strategies for its assimilation. Thus, for example, [6] verified through a research study carried out on 522 secondary school students in Cuba that students have unproductive beliefs about the learning of this science, hence affecting the teaching results; that is, the concept that mathematics is a difficult subject affects their performance negatively. Students also believe that mathematics is useful, a perception positively related with performance, as it is the most influential factor in the level of interest shown in this discipline and, therefore, in its learning [36].
(c)
Students express they receive continuous messages from their environment that influence their beliefs about mathematics and learning mathematics. Beliefs in the social context refer to the role and function of teachers, the role and function of students in their own classroom, and the socio-mathematical rules and practices in the classroom. For this reason, [37] points out that there is a strong link between the teachers’ beliefs and students within the school environment.
Regarding the beliefs on the role of teachers and their methodology in learning, there are two well-differentiated points of view. According to the first, many students perceive their mathematics teacher as a purveyor of knowledge, from whom they need to learn everything he or she conveys. The second point of view has a constructivist character, and considers the teacher not as an informer, but as a driver who facilitates students’ learning by giving meaning to what they are learning. The teacher establishes an affective relationship with his or her students, paying attention to their needs and opinions, taking into account their diversity, fostering interaction with students [7]. In this sense, [38] defend a model of prosocial classroom in which the social and emotional competence of teachers is of great importance in generating healthy relationships with their students and, consequently, in creating a good classroom atmosphere.
Different studies also investigated the beliefs of students regarding mathematics according to their age [39,40] in particular, assert that there is a set of negative beliefs that get reinforced during secondary education, compared to primary education that doubtlessly influence the negative outcomes found. The emergence of these beliefs throughout secondary education is related to the fact that mathematics is an increasingly difficult subject that causes unsatisfactory academic performance [8,41].
Previous investigations seem to show the importance of beliefs about mathematics and learning mathematics for comprehension and learning optimization, and also in fostering students’ motivation [24]. However, so far, no studies have been undertaken in which possible relations between the three categories of beliefs described and motivation are established, differentiating the four years of compulsory secondary education, in a medium-low level social context. For this reason, this paper delves into the possible relationship existing between differences in students’ beliefs according to their academic level and their level of motivation.
With this aim, the specific objectives are as follows:
  • To study the level of motivation towards mathematics of a sample of secondary school students according to their academic level in a medium-low level social context.
  • To analyse the possible differences between types of beliefs linked to learning mathematics according to the academic level in a medium-low level social context.
  • To establish possible connections between the level of motivation that the students in the sample show towards mathematics and the beliefs they express regarding the process of teaching and learning mathematics.

2. Method

2.1. Context Characteristics

The law in force in Spain since 2013 regulating the Spanish education system is the Organic Law for the Improvement of Educational Quality (LOMCE) [42], which determines 65% of the curriculum of the different Autonomous Communities. In the case of the Basque Country, there is also Heziberri 2020 [43] which is a plan of the Department of Education of the Basque Government for the improvement of the education system of the said Autonomous Community.
According to the regulations of the LOMCE and Heziberri 2020 Mathematics is a core subject; that is, its teaching schedule occupies at least 50% of the total and is also mandatory during the four years of Compulsory Secondary Education (ESO). This educational stage is characterized by being compulsory and free and by covering students aged 12 to 16.
The curricular content is distributed into four blocks in each of the four ESO years: 1. Numbers and Algebra, 2. Geometry and Measurement, 3. Functions and Graphs, 4. Statistics and Probability, so that from 3rd year onwards content of a more abstract nature is taught. As a possible consequence, the PISA results (2018) for Spain showed the difficulties of interpretation, communication, and mathematical abstraction in 15-year-old students (3rd ESO), at the very time which the limit associated with arithmetic and geometric progressions is introduced along with other abstract content. In Spain, only 7% of students of this age reach levels 5 and 6 (levels that include abstract content) of mathematics in this test, the average of the OECD countries being 11% [44].
The school the sample belongs to is located in a town of approximately 30,000 inhabitants within the area of Greater Bilbao (the Basque Country) in northern Spain, on the eastern shore of the Cantabrian sea. Consequently, the educational planning and curricular design of the school mentioned are consistent with the two educational laws and regulations, LOMCE and Heziberri 2020, in force in the Basque Autonomous Community (Spain). Specifically, all students who are in the same year receive the same mathematical training, since there are no different levels in the same classroom. As for the teachers, six different participants who were likely to teach in different levels, participated in the study. The teaching methodology employed by the teachers involved in the study was mostly based on oral explanations.
With regard to the socioeconomic level, this town has an unemployment rate of 21.67%, double the average rate for the Basque Country; half of the population have had only primary education or no formal education. The per capita GDP is 17,570 USD, while the Basque Country per capita GDP for 2018 was 31,588 USD [45]. Therefore, we can say that the school belongs to a medium-low level social context.

2.2. Sample

The sample examined in this study consisted of 202 secondary school students, 86 girls and 116 boys, belonging to the same public school. In terms of the distribution of the students by academic years, 32.2% of them attended the first level in secondary education (aged between 12 and 13); 27.3% were in the second level (aged between 13 and 14) and 12.9% in the third level (ranging in age from 14 to 15 years). Finally, the participants in the fourth level in secondary education (aged 15–18) were 27.8% in the sample.

2.3. Measuring Instruments and Procedure

The students in the sample completed two surveys in the third trimester before their final exams, with the purpose of evaluating their motivation and their beliefs towards learning mathematics. The questionnaires used in this study were developed with samples of a cultural context similar to that of the sample involved in this research project. This was the main reason for selecting these questionnaires for the present study.
Motivation was evaluated through a five-point Likert-type questionnaire consisting of 26 statements and designed by [46]. The statements in this text are related to essential aspects of motivation, such as interest and enthusiasm towards academic activity, students’ aspirations and their feeling of independence, their needs and their opinions related to the usefulness of the subject; participants were asked to express to what extend they agreed with each of the statements.
Students’ beliefs about learning mathematics were evaluated using the questionnaire by [40]. This test consisted of 41 questions related to school learning processes of mathematics in which participants have to express agreement or disagreement through a five-point Likert-type scale; where 1, is strongly disagree and 5, completely agree. The categories used in this questionnaire, together with some examples of each one, are described in Table 1:
The board of the school participating in the study were informed and gave consent for the research. Later, parents and teachers of the students involved also gave their informed consent. Questionnaires were filled in anonymously, in two sessions during school hours in March 2016. Two members of the research team were responsible for their administration.
Regarding statistical procedures, the differences related to the sex variable were evaluated through the Mann-Whitney U test [47] and the Pearson correlation coefficient was used to examine the size effect [48].

2.4. Validity and Reliability

The scale-level-CVI (S-CVI) was the parameter used to quantify the content validity of each of the surveys involved in the present research project [49,50]. To that end, a panel of experts was invited to assess both the questionnaire of Motivation in Mathematics (QMM) [48] and the questionnaire about Mathematical Beliefs (QMB) [39]. More specifically, twelve professionals in the field of education received an invitation to evaluate the items in the survey, among which nine agreed to participate in the evaluation process.
The criteria to choose the experts included in this panel were, firstly, to have no connection with the research project being undertaken and, secondly, to have an acknowledged expertise in the field of didactics of mathematics, science education or educational psychology. Some teachers with a longstanding experience in teaching in compulsory education were also invited to participate in the pool of evaluators. Table 2 presents a description of the experts who form the cast of experts that assessed the suitability of the items included in the two questionnaires used in the study.
The panel of experts were asked to fulfil two formularies, one for each of the two questionnaires of the present study. At the beginning of each of the formularies, a short description concerning the purpose of the survey to be assessed was introduced to the evaluators. Furthermore, the experts were asked by means of a 5-point Likert scale to assess the appropriateness of each of the items to measure the motivation towards learning mathematics in secondary education (in the case of the QMM questionnaire) as well as teenagers’ beliefs on learning mathematics (in the case of QMB questionnaire). The scale ranged from 1 (the item presented is largely irrelevant for the objective of the research) to 5 (the item presented is very suitable for the objective of the research).
Building on the responses collated, the item-rated content validity index (I-CVI) was calculated by dividing the number of experts that allocated a grade equal or higher than three to each item by the total number of assessments that each item received. The I-CVI limit value for considering an excellent level of agreement usually stands at 0.77, which in this particular case, entails that at least 7 out of the 9 experts allocated an appropriate assessment to the item in question. The average of the I-CVI values of all the items included in each of the surveys allowed us to determine the scale-level-CVI (S-CVI). The minimum value for reflecting an adequate content validity of the latter statistical indicator is 0.8 [50]. Table 3 introduces the results of the aforementioned study. The results reached indicated that the S-CVI of the two surveys surpasses the cut-off value of 0.78, showing an adequate level of content validity [51].
Finally, the Cronbach’s alpha was used to estimate the reliability of the surveys [46]. Table 4 displays the information regarding the results of this study. It is worth noting that the results presented in this study essentially matched those initially published by the authors of the surveys [35,43], and they allow us to assume that both questionnaires display an adequate level of internal consistency.

3. Results

Table 5 shows the results obtained for motivation towards mathematics in each academic level analysed and their corresponding significance, using the Kruskal-Wallis contrast test:
Results show that the best motivation for learning mathematics is obtained by 1st-year students, and the worst by 3rd-year students. Once the corresponding post hoc analysis of the average differences between academic levels was performed, the differences between 3rd-year students and all the rest were taken as significant: 4th (p < 0.01), 2nd (p < 0.001) and 1st (p < 0.001).
Table 6 below shows the results obtained for beliefs towards mathematics according to academic level and the corresponding significance, using the Kruskal-Wallis contrast test.
In 11 out of the 42 beliefs the questionnaire consists of, results show significant differences according to the students’ academic level, which are shown in Table 6. Out of these 11 beliefs, eight are related to the classroom context and more specifically to the teachers: the first four (beliefs 2, 19, 22, and 25) are related to their personal characteristics, such as kindness, friendliness, confidence in their students, belief 21 to their teaching methods and the last three are related to their capacity to help.
Related to belief 2, significant differences were found between 1st and 3rd-year students (p < 0.01) and between 1st and 4th-year students (p < 0.01), in the sense that 1st-year students express with had good mathematics teachers, a belief that decreases considerably in the 3rd and 4th years.
The results obtained for beliefs 19, 22 and 25 indicate that the worst view of teachers is perceived in 3rd year. The significant differences obtained for belief 19 indicate that 3rd-year students consider that teachers have an important influence on their performance in mathematics, as there were differences between 1st and 3rd years (p < 0.01), between 2nd and 3rd years (p < 0.01) and between 3rd and 4th years (p < 0.05. At the same time, the results obtained for belief 22 show that 3rd-year students have less friendly relationships with mathematics teachers than students in other years: 1st (p < 0.01), 2nd (p < 0.01) and 4th (p < 0.01). Along the same lines, 3rd-year students are the ones who consider that mathematics teachers are not confident in the performance of large numbers of their students (belief 25).
Among the results obtained for the beliefs on the classroom context, what stands out is that beliefs 21 and 36 present the greatest size effect. Again, it is 3rd-year students who consider that teaching methods are more boring, with significant differences with 1st year (p < 0.001), 2nd year (p < 0.001) and 4th year (p < 0.001); this group are also the ones who believe that mathematics teachers are less willing to clarify doubts (p < 0.001).
Regarding the beliefs related to the student, numbers 16 and 17 show significant differences. In both cases, 4th-year students admit to a larger degree their lack of study and effort, and attribute their failure in mathematics to their own lack of capability.
Lastly, for belief number 5 about mathematics, 3rd and 4th-year students attribute their difficulties in mathematics to the subject’s own difficulty, to a larger extent than 1st and 2nd-year students (p < 0.01 and p < 0.001 respectively).
Regarding the relationship between the level of motivation and beliefs towards mathematics by the students in the sample, we can verify that the students show lower motivation in 3rd year, coinciding with the fact that beliefs in the three categories in this year are more negative.

4. Conclusions

The data obtained seems to indicate that the students in the sample show different levels of motivation and beliefs towards learning mathematics according to their academic year.
Regarding the first objective referring to motivation, we find the lowest levels in the later academic years, 3rd year being especially noticeable, while 1st-year students show a higher motivation. Indeed, 1st-year students start off motivated to learn the subject, then their motivation decreases slightly in 2nd year, declines abruptly in 3rd year, and a slight recovery is appreciated in 4th year, but without reaching the levels of 2nd year.
These results are along the same lines as those obtained in previous research [39,43]. Specifically, [39] studied a sample of students between the ages of 11 and 18, according to which the affective factors involved in learning mathematics are less favourable as the academic level advances.
As we understand, there are four main reasons that can explain these results, not in order of importance:
(a)
The use of classroom methodologies that foster students’ passivity.
(b)
The type of content in 3rd year and 4th year are more abstract.
(c)
A decrease in the self-perception of competence in mathematics.
(d)
The evolutionary characteristics of adolescence.
(a) The first reason given to justify the results obtained for motivation is linked to the use of classroom methodologies and more specifically to the teachers [47] as we explained in the context characteristics, or at least, this is how students seem to experience methodologies.
This interpretation of results coincides in turn with those from other research studies, which state that by using cooperative and active techniques, students feel more secure than working on their own, as interaction and a liking for mathematics are promoted [4,48], which logically should entail a higher level of motivation. With this in mind, [48] carried out an intervention with secondary school students in the Milan area so that one group of students received innovative instruction, while the other received traditional instruction, being able to verify that the first group manifested better beliefs towards mathematics.
As a consequence, it is imperative to review mathematics teacher training, so that their educational task is not essentially based on the students assimilating theoretical concepts in a passive manner. They should also be able to transmit knowledge in an intelligible manner to their students, “connecting” with them to promote greater active involvement in learning mathematics, based on their previous knowledge of the subject, their interests, needs and, very importantly, believing in them and their potential.
(b) The second explanatory reason we introduced related to the increase in the degree of abstraction in mathematics in the third year, together with an insufficient methodological adequacy in this circumstance, could help us understand the reason for the dramatic decrease in motivation in third year. Indeed, we already mentioned in the “Context characteristics” section that in the mathematics curriculum of the ESO, content of a more abstract nature is taught from the third year onwards, and how this seems to have had an impact on the latest results obtained in the PISA report (2018). In addition, more appropriate active methodologies addressing this increase in the degree of abstraction of this subject do not occur.
(c) The third explanatory cause of the results obtained for motivation in this research related to a decrease in the self-perception of competence in mathematics [5,37], could somehow be considered to be a consequence of (a) and (b) as previously mentioned and above all, of the difficulty derived from the increase in the degree of abstraction of mathematics. As [10] asserts, we agree on the importance of confidence in the readiness and ability to learn mathematics in order to have the right motivation towards this discipline, i.e., to have a more intrinsic motivation. The subjects’ active involvement in the learning process increases when they feel competent, i.e., when they trust their own capabilities and have high expectations of self-efficacy [52].
In the end, this could be the explanation why 4th-year students get the second worst results in motivation. Since these students attribute their poor results to their lack of effort and capabilities to a larger extent, they are more aware that their personal involvement in learning mathematics results in their performance, which at the same time encompasses an increase in motivation from the previous year.
(d) Finally, we believe that it is necessary to take into account the characteristics of this age range to understand the changes in the level of motivation that we obtained in our results. Along the same lines as other research studies, our results suggest that as secondary education students advance along their academic path, their beliefs and perspectives evolve too, in a way they shift from being more innocent and simplistic to being more realistic and complex [20,39,53].
Although an increase in the complexity of thought would seem to point in the direction of greater capacity to undertake a higher degree of abstraction, and therefore complexity in mathematics, this could be the reason students in the first two years have a more positive view of learning mathematics with higher motivation levels, while in the third year and thereafter they become more realistic and include more explanatory variables into their performance.
Regarding the second objective on the possible differences between types of beliefs linked to learning mathematics according to the academic level, again, we verify that the most differentiated beliefs are mainly those of 3rd-year students compared to the rest of the levels. Below we see the main conclusions based on the three types of beliefs analysed (see Table 6).

4.1. Beliefs on the Classroom Context

We first focused on those that refer to the way of experiencing the methodology of the teaching staff. An aspect related to this methodology and specifically to the educational relationship with the teachers within the classroom, is the help given by the teacher, or it would be better to say, the experience that students have in this regard. This subjective interpretation by students showed significant differences in our study in the case of beliefs 19, 25, 6, 14 and 36 (see Table 6). In particular, it can be verified that the results obtained for these beliefs indicate that the worst view of teachers’ methodology is perceived in 3rd year. In this sense, we concur with [7] on the importance of teacher-student communication in the mathematics classroom to clarify doubts and help with difficulties.
In the sphere of the educational relationship with the teachers which is closely related to the methodology they use, we also deem it relevant to reflect on the significant differences in the students’ experiences according to the year, reflected also in their “beliefs on the classroom context” related to their personal relationship with their teachers. Again, it is 3rd-year students who present the worst results, but the following beliefs especially attract our attention (see Table 6): “I have almost always had good mathematics teachers” (belief 2), “My relationship with mathematics teachers is more friendly than that with teachers of other subjects” (belief 22) and “I think some mathematics teachers do not trust the performance of large numbers of their students” (belief 25).

4.2. Beliefs about Oneself as a Learner

In fourth year, beliefs attributing poor results in mathematics to the lack of one’s own study and effort (see Table 6, belief 16) and even one’s own capabilities (see Table 6, belief 17), seem to be stronger. We understand that these results could be related to the decrease in the self-perception of competence in mathematics [21,29], which in turn could be considered to be a consequence of the progressive increase in the degree of abstraction of mathematics.
These results seem to confirm the model proposed by [54], in which this author suggested that students’ beliefs lead to affective-emotional responses that have repercussions in the perception of the discipline, their self-concept, and consequently, their performance and expectations of success. However, it is noteworthy that in the third year these beliefs obtain lower scores, which suggests the need to deepen these results in future research to assess whether there could be other variables involved in addition to the degree of abstraction of mathematics.

4.3. Beliefs about Mathematics

Within this group of beliefs, the only one to obtain significant differences was number 5, in which students attribute their difficulties in mathematics to the subject’s own difficulty. In this case, 1st-year students are the ones who attribute difficulties to a lesser extent, probably because they are the ones who have less perception of difficulty, the contents of that year with a lower degree of abstraction.
Regarding the third objective of our research, we could link the lowest level of motivation manifested in 3rd year of ESO to the results obtained by these students on their beliefs. These results show that 3rd-year students present the worst results and the greatest differences when comparing academic levels. Specifically, the results obtained for the belief “The methods used by mathematics teachers are more boring than those used by teachers of other subjects”, which showed a greater size effect in the difference between 3rd-year students and the rest, seem to confirm the idea that motivation towards mathematics is closely related to the methodology used in the classroom.
In relation to the category of beliefs that we called “beliefs on the classroom context”, data seem to indicate that students manifesting the most negative beliefs are those in 3rd year, a time when we already know students also show the lowest level of motivation. This allows us to affirm that a passive and unattractive methodology is related with lower motivation, which in turn, has repercussions in manifesting unfavourable beliefs about the educational relationship within the classroom. Indeed, these beliefs of students about their teachers, although maybe not a triggering factor for rejection by themselves, could certainly contribute to students with lower motivation towards mathematics (especially the results obtained in belief 25).
Nevertheless, it still does not explain the reason this situation improves in the fourth year and worsens specifically in the third year. The differences in motivation between third and fourth years could be explained from the perspective that in third year the tendency is to attribute difficulties to external factors, especially those related with the teachers, while in the fourth year, as the elaboration of their self-identity has advanced, beliefs related to students themselves appear as more significant. This explains the different evolution of motivation when linked to beliefs where the individual is in less control and depends more on the teachers, or when linked to beliefs where individuals have it in their power to improve. This issue seems worth delving deeper into in future research.
Regarding the “Beliefs about oneself as a learner”, the decrease in the self-perception of competence in mathematics [8,43], could somehow be considered to be a consequence of the difficulty derived from the increase in the degree of abstraction of mathematics, which in turn is related, as we already mentioned, with a decrease in motivation.
Regarding the changes in motivation related to the characteristics of this adolescent stage, older students give more importance to beliefs related to the school context and even themselves, all of which are reflected in their motivation. In any case, we should not forget that teachers, according to the way they develop their work, also have an influence on the construction of beliefs towards the subject they teach, and hence on motivation [55].
Ultimately, it all seems to indicate that the types of beliefs that seem to be related with motivation show a greater impact in the second half of Compulsory Secondary Education (3rd and 4th years). This result shows the strong tie existing between motivation and beliefs, and therefore, their importance in mathematics education. It seems reasonable to think that teachers are important at all levels, and one of their main challenges for the future in the teaching of mathematics is not just the transferring of knowledge, but a great part of their task is building up enthusiasm for mathematics [56]. However, it also seems necessary to keep in mind the view students have on the subject and its learning process, which is clearly manifested in the beliefs and in relation to the corresponding motivation.
Moreover, it will be especially necessary to pay more attention to 3rd and 4th-year students to design educational interventions that further take into account the types of limiting beliefs in each of these levels that we have referred to, as they seem to have repercussions on students’ motivation.
Finally, it is necessary to take into consideration that this study has some limitations such as the sample size, and that it should be extended to different contexts. We understand that in the social context of this research, education is valued as a social ladder and that mathematics is a very important part of the curriculum as we have pointed out. All of this influences social beliefs towards mathematics, which are seen as important and difficult at the same time. That is the reason the research was done in a single school—to try and homogenize these social beliefs to the fullest, although it would be very interesting to compare with what might happen in other types of contexts. Finally, it would also be interesting in future research to contrast the beliefs of students towards mathematics with those of other schools, where they work with more active methodologies.

Author Contributions

Conceptualization, resources, project administration, J.M.M.; methodology, J.M.M., V.R.R. and J.D.V.; validation, formal analysis, J.D.V.; investigation, data curation, writing–original draft preparation, writing–review and editing, V.R.R.; visualization, supervision, J.M.M. and V.R.R.; funding acquisition, J.M.M. and J.D.V. All authors contributed equally to this work. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the University of the Basque Country through the project EHU 15/19 and The APC was funded by the University of the Basque Country through the project PES17/39.

Conflicts of Interest

The authors declare no conflict of interest

References

  1. Goleman, D. Inteligencia Emocional; Kairós: Barcelona, Spain, 1997. [Google Scholar]
  2. Leonard, M.J.; Kalinowski, S.T.; Andrews, T.C. Misconceptions yesterday, today, and tomorrow. Cbe Life Sci. Educ. 2014, 13, 179–186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Nyberg, E.; Brkovic, I.; Sanders, D. Beauty, memories and symbolic meaning: Swedish student teachers views of their favourite plant and animal. J. Biol. Educ. 2019, 1–14. [Google Scholar] [CrossRef] [Green Version]
  4. Donvito, A.; Otero, M.R. Why do adults go to adult secondary school? Ikastorratza. E Rev. De Didáctica 2019, 23, 81–93. [Google Scholar] [CrossRef]
  5. Gafoor, K.A.; Kurukkan, A. Why High School Students Feel Mathematics Difficult? An Exploration of Affective Beliefs. Online Submiss 2015. Available online: https://eric.ed.gov/?id=ED560266 (accessed on 1 January 2020).
  6. Vizcaino, A.E.; Manzano, M. Análisis de las relaciones entre las creencias epistemológicas sobre la matemática y rendimiento académico. Psychol. Soc. Educ. 2017, 9, 105–119. [Google Scholar] [CrossRef] [Green Version]
  7. Chacón, I.M.G. Matemática Emocional: Los Afectos en el Aprendizaje Matemático; Narcea Ediciones: Madrid, Spain, 2000; Volume 83. [Google Scholar]
  8. Alonso, S.H.; Sáez, A.M.; Picos, A.P. ¿Por qué se rechazan las matemáticas? Análisis evolutivo y multivariante de actitudes relevantes hacia las matemáticas. Rev. Educ. 2004, 334, 75–95. [Google Scholar]
  9. McLeod, D.B. Affective issues in mathematical problem solving: Some theoretical considerations. J. Res. Math. Educ. 1988, 19, 134–141. [Google Scholar] [CrossRef]
  10. McLeod, D.B. Research on affect in mathematics education: A reconceptualization. Handb. Res. Math. Teach. Learn. 1992, 1, 575–596. [Google Scholar]
  11. Lomas, G.; Grootenboer, P.; Attard, C. The affective domain and mathematics education. In Research in mathematics education in Australasia 2008–2011; Springer: Berlin/Heidelberg, Germany, 2012; pp. 23–37. [Google Scholar]
  12. Attard, C.; Ingram, N.; Forgasz, H.; Leder, G.; Grootenboer, P. Mathematics education and the affective domain. In Research in Mathematics Education in Australasia 2012–2015; Springer: Berlin/Heidelberg, Germany, 2016; pp. 73–96. [Google Scholar]
  13. Di Martino, P.; Zan, R. ‘Me and maths’: Towards a definition of attitude grounded on students’ narratives. J. Math. Teach. Educ. 2010, 13, 27–48. [Google Scholar] [CrossRef]
  14. Ricoy, M.; Couto, V.S.; João, M. Desmotivación del alumnado de secundaria en la materia de matemáticas. Rev. Electrónica Investig. Educ. 2018, 20, 69–79. [Google Scholar] [CrossRef]
  15. Alsina, Á.; Domingo, M. Cómo aumentar la motivación para aprender matemáticas. Suma 2007, 56, 23–31. [Google Scholar]
  16. Brophy, J. Motivating Students to Learn; Routledge: Abingdon, UK, 2013. [Google Scholar]
  17. Ortíz, C.V.; Pastells, ÁA. Un modelo para el análisis de objetos matemáticos en libros de texto chilenos: Situaciones problemáticas, lenguaje y conceptos sobre probabilidad. Profe. Rev. Currículum Form. Profe. 2015, 19, 441–462. [Google Scholar]
  18. Tella, A. The impact of motivation on students’ academic achievement and learning outcomes in mathematics among secondary school students in Nigeria. Eurasia J. Math. Sci. Technol. Educ. 2007, 3, 149–156. [Google Scholar] [CrossRef]
  19. Yunus, M.; Ali, W.Z.W. Motivation in the Learning of Mathematics. Eur. J. Soc. Sci. 2009, 7, 93–101. [Google Scholar]
  20. Cano, F. Epistemological beliefs and approaches to learning: Their change through secondary school and their influence on academic performance. Br. J. Educ. Psychol. 2005, 75, 203–221. [Google Scholar] [CrossRef]
  21. Beghetto, R.A.; Baxter, J.A. Exploring student beliefs and understanding in elementary science and mathematics. J. Res. Sci. Teach. 2012, 49, 942–960. [Google Scholar] [CrossRef]
  22. Stevens, T.; Olivarez, A.; Lan, W.Y.; Tallent-Runnels, M.K. Role of mathematics self-efficacy and motivation in mathematics performance across ethnicity. J. Educ. Res. 2004, 97, 208–222. [Google Scholar] [CrossRef]
  23. Skaalvik, E.M.; Federici, R.A.; Klassen, R.M. Mathematics achievement and self-efficacy: Relations with motivation for mathematics. Int. J. Educ. Res. 2015, 72, 129–136. [Google Scholar] [CrossRef]
  24. Kloosterman, P. Beliefs about mathematics and mathematics learning in the secondary school: Measurement and implications for motivation. In Beliefs: A Hidden Variable in Mathematics Education? Springer: Berlin/Heidelberg, Germany, 2002; pp. 247–269. [Google Scholar]
  25. Pintrich, P.R.; Schunk, D.H.; Luque, M.L. Motivación en Contextos Educativos: Teoría, Investigación y Aplicaciones; Pearson Prentice Hall: Madrid, Spain, 2006. [Google Scholar]
  26. Pintrich, P.R.; De Groot, E.V. Motivational and self-regulated learning components of classroom academic performance. J. Educ. Psychol. 1990, 82, 33. [Google Scholar] [CrossRef]
  27. Anderman, E.M.; Dawson, H. Learning with motivation. In Handbook of Research on Learning and Instruction; Mayer, R.E., Alexander, P.A., Eds.; Routledge: Abingdon, UK, 2011; pp. 219–241. [Google Scholar]
  28. Retelsdorf, J.; Köller, O.; Möller, J. On the effects of motivation on reading performance growth in secondary school. Learn. Instr. 2011, 21, 550–559. [Google Scholar] [CrossRef]
  29. Awan, R.; Noureen, G.; Naz, A. A Study of Relationship between Achievement Motivation, Self Concept and Achievement in English and Mathematics at Secondary Level. Int. Educ. Stud. 2011, 4, 72–79. [Google Scholar] [CrossRef] [Green Version]
  30. Goldin, G.; Rösken, B.; Törner, G. Beliefs–no longer a hidden variable in mathematical teaching and learning processes. In Beliefs and Attitudes in Mathematics Education; Maass, J., Schlöglmann, W., Eds.; Brill Sense: Rotterdam, The Netherlands, 2009; pp. 1–18. [Google Scholar]
  31. De Corte, E.; Mason, L.; Depaepe, F.; Verschaffel, L. Self-regulation of mathematical knowledge and skills. In Handbook of Self-Regulation of Learning and Performance; Schunk, D.H., Zimmerman, B., Eds.; Taylor & Francis: Abingdon, UK, 2011; pp. 155–172. [Google Scholar]
  32. Pajares, F. Motivational role of self-efficacy beliefs in self-regulated learning. In Motivation and Self-Regulated Learning: Theory, Research, and Applications; Taylor and Francis: Abingdon, UK, 2008; pp. 111–139. [Google Scholar]
  33. Gil, N.; Blanco, L.; Guerrero, E. El dominio afectivo en el aprendizaje de las matemáticas. Una revisión de sus descriptores básicos. Rev. Iberoam. Educ. Matemática 2005, 2, 15–32. [Google Scholar]
  34. Chen, J.A.; Pajares, F. Implicit theories of ability of Grade 6 science students: Relation to epistemological beliefs and academic motivation and achievement in science. Contemp. Educ. Psychol. 2010, 35, 75–87. [Google Scholar] [CrossRef]
  35. Schommer-Aikins, M.; Duell, O.K.; Hutter, R. Epistemological beliefs, mathematical problem-solving beliefs, and academic performance of middle school students. Elem. Sch. J. 2005, 105, 289–304. [Google Scholar] [CrossRef]
  36. Schommer-Aikins, M.; Duell, O.K. Domain specific and general epistemological beliefs. Their effects on mathematics. Rev. Investig. Educ. 2013, 31, 330. [Google Scholar]
  37. Parra, H. Creencias matemáticas y la relación entre actores del contexto. Rev. Latinoam. Investig. Matemática Educ. Relime 2005, 8, 69–90. [Google Scholar]
  38. Jennings, P.A.; Greenberg, M.T. The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Rev. Educ. Res. 2009, 79, 491–525. [Google Scholar] [CrossRef]
  39. Hidalgo, S.; Maroto, A.; Ortega, T.; Palacios, A. Atribuciones de afectividad hacia las Matemáticas. Unión. Rev. Iberoam. Educ. Matemática 2013, 35, 93–113. [Google Scholar]
  40. Chen, P.; Zimmerman, B. A cross-national comparison study on the accuracy of self-efficacy beliefs of middle-school mathematics students. J. Exp. Educ. 2007, 75, 221–244. [Google Scholar] [CrossRef]
  41. Hidalgo, S.; Maroto, A.; Palacios, A. El perfil emocional matemático como predictor de rechazo escolar: Relación con las destrezas y los conocimientos desde una perspectiva evolutiva. Educ. Matemática 2005, 17, 89–116. [Google Scholar]
  42. Ley Orgánica 8/2013, de 9 de diciembre, para la Mejora de la Calidad Educativa (LOMCE), Boletín Oficial del Estado (BOE). 10 de diciembre de 2013. Available online: https://www.boe.es/buscar/pdf/2013/BOE-A-2013-12886-consolidado.pdf (accessed on 2 December 2019).
  43. Government, Basque. Decree 236/2015, by which the Curriculum of Basic Education is Established and Implemented in the Basque Autonomous Community (15th of January 2016 EHAA); Government, Basque: Vitoria-Gasteiz, Spain, 2015. [Google Scholar]
  44. Educacionyfp.gob.es. PISA 2018 [en línea]. 2019. Available online: http://www.educacionyfp.gob.es/inee/evaluaciones-internacionales/pisa/pisa-2018.html (accessed on 15 December 2019).
  45. EUSTAT Realidad estadística actual y evolución histórica de Sestao. Available online: http://www.eustat.eus/municipal/datos_estadisticos/sestao_c.html (accessed on 2 December 2019).
  46. Rey, J.M.; Hidalgo, E.; Espinosa, C. La motivación en la escuela: Cuestionarios para su análisis. In Temas de Educación; Librería Ágora: Málaga, Granada, Spain, 1989. [Google Scholar]
  47. Siegel, S.; Castellan, N.J. Nonparametric Systems for the Behavioural Sciences; McGraw Hill: New York, NY, USA, 1988. [Google Scholar]
  48. Tomczak, M.; Tomczak, E. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends Sport Sci. 2014, 21. [Google Scholar]
  49. Bolarinwa, O.A. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger. Postgrad. Med. J. 2015, 22, 195. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Larsson, H.; Tegern, M.; Monnier, A.; Skoglund, J.; Helander, C.; Persson, E.; Malm, C.; Broman, L.; Aasa, U. Content validity index and intra-and inter-rater reliability of a new muscle strength/endurance test battery for Swedish soldiers. PLoS ONE 2015, 10, e0132185. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Rodrigues, I.B.; Adachi, J.D.; Beattie, K.A.; MacDermid, J.C. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. Bmc Musculoskelet. Disord. 2017, 18, 540. [Google Scholar] [CrossRef] [Green Version]
  52. Pérez, J.C.N.; González-Pienda, J.A. Determinantes del Rendimiento Académico: (Variables Cognitivo-Motivacionales, Atribucionales, Uso de Estrategias y Autoconcepto); Universidad de Oviedo: Oviedo, Spain, 1994. [Google Scholar]
  53. Mason, L.; Scrivani, L. Enhancing students’ mathematical beliefs: An intervention study. Learn. Instr. 2004, 14, 153–176. [Google Scholar] [CrossRef]
  54. Mason, L. High school students’ beliefs about maths, mathematical problem solving, and their achievement in maths: A cross-sectional study. Educ. Psychol. 2003, 23, 73–85. [Google Scholar] [CrossRef]
  55. McLeod, D.B. Beliefs, attitudes, and emotions: New views of affect in mathematics education. In Affect and Mathematical Problem Solving; Springer: Berlin/Heidelberg, Germany, 1989; pp. 245–258. [Google Scholar]
  56. Alsina, C. Mañana será otro día: Un reto matemático llamado futuro. Goñi Jm Currículum Matemáticas Los Inicios Siglo 2000, 21, 13–21. [Google Scholar]
Table
Table 1. Description of the categories of beliefs.
Table 1. Description of the categories of beliefs.
CategoriesExamples of Beliefs
Beliefs about the classroom’s specific context
  • Mathematics teachers’ methods are usually more boring than those of other subjects.
  • Mathematics teachers are always willing to clarify doubts and difficulties.
Beliefs about mathematics
  • Mathematics is very abstract and far from reality.
  • The difficulties I have or may have in mathematics are due to the subject’s difficulties.
Beliefs about oneself as a learner of mathematics
  • When I get good marks in mathematics it is mainly thanks to my capability
  • Mathematics is a challenge to one’s own capability.
Table
Table 2. Composition of the panel of experts that reviewed the appropriateness of the surveys involved in the study.
Table 2. Composition of the panel of experts that reviewed the appropriateness of the surveys involved in the study.
Academic LevelScope of Expertise Professional ActivityN
PhDDidactic of mathematicsLectures at university level3
Science education3
Educational psychology2
Bachelor’s degreeCompulsory educationTeacher1
Table
Table 3. The outcomes of the study of the item-rated content validity indices (I-CVI) and scale-level-CVI (S-CVI) in the questionnaire of Motivation in Mathematics (QMM) and in the questionnaire about Mathematical Beliefs (QMB).
Table 3. The outcomes of the study of the item-rated content validity indices (I-CVI) and scale-level-CVI (S-CVI) in the questionnaire of Motivation in Mathematics (QMM) and in the questionnaire about Mathematical Beliefs (QMB).
SurveyObjectiveExpert PanelItems% Items with Agreement (>7)% Items I-CVI > 0.77S-CVI
QMMMotivation towards mathematics92696.296.20.93
QMBBeliefs regarding mathematic94493.2910.89
Table
Table 4. The outcomes of the study as to the reliability of Questionnaire of Motivation in Mathematics (QMM) and Questionnaire about Mathematical Beliefs (QMB).
Table 4. The outcomes of the study as to the reliability of Questionnaire of Motivation in Mathematics (QMM) and Questionnaire about Mathematical Beliefs (QMB).
SurveyItemsCronbach’s Alpha
QMM260.93
QMB440.88
Table
Table 5. Motivation towards mathematics according to the students’ academic level.
Table 5. Motivation towards mathematics according to the students’ academic level.
ScaleMedianMean (SD)
Motivation 1948666.37894.1(15.7)88.5(19.2)68(17.1)80(16.4)
1st2nd3rd4th1st2nd3rd4th
1 Kruskal-Wallis H-test = 36.1 [3], p < 0.001, η2 = 0.206. N = 176.
Table
Table 6. Beliefs about mathematics with significant differences according to academic level, grouped into the three categories described.
Table 6. Beliefs about mathematics with significant differences according to academic level, grouped into the three categories described.
BeliefMedianMean (SD)
Beliefs about the Classroom Context
1 2—I have almost always had good mathematics teachers 32222.8 (0.9)2.6 (0.9)2.1 (1.1)2.3 (1)
2 19—My performance in mathematics depends on the teacher to a large extent1.51321.6 (0.9)1.5 (1)2.5 (1.4)1.7 (1.2)
3 22—My relationship with mathematics teachers is more friendly than that with teachers of other subjects11111.4 (1.1)1.3 (0.8)0.7 (0.8)1.4 (1)
4 25—I think some mathematics teachers do not trust the performance of large numbers of their students 11.1321.2 (1.1)1.8 (1.1)2.5 (1.3)1.8 (1.4)
5 21—The methods used by mathematics teachers are more boring than those used by teachers of other subjects 12311.4 (1.2)1.8 (1)2.7 (1)1.7 (1.1)
6 6—The difficulties I have or may eventually have in mathematics are due to a lack of help1121.51 (0.9)1 (0.9)1.7 (1.3)1.8 (1.2)
7 14—When I get poor marks in mathematics it is due mainly to the teachers’ lack of help1.31211 (1.1)1 (1)2 (1.5)1.3 (1)
8 36—Mathematics teachers are always willing to clarify doubts and difficulties in the classroom33033.1 (1.1)2.8 (1)1.1 (1.5)2.6 (1.2)
1st2nd3rd4th1st2nd3rd4th
Beliefs about oneself as a Learner
9 16—When I get poor marks in mathematics it is mainly due to my lack of effort and study22231.9 (1.3)2.1 (1.3)1.8 (1.3)2.8 (0.9)
10 17—When I get poor marks in mathematics it is mainly due to my lack of capability11121.3 (1.2)1.3 (1.3)0.7 (0.7)1.9 (1.2)
1st2nd3rd4th1st2nd3rd4th
Beliefs about Mathematics
11 5—The difficulties I have or may eventually have in mathematics are due to its difficulty12221.4 (1)2.1 (1.2)2.2 (1.1)2.3 (1.2)
1st2nd3rd4th1st2nd3rd4th
1 Kruskal-Wallis H-test = (3, N = 181) = 11.8, p < 0.01, η2 = 0.07; 2 Kruskal-Wallis H-test = (3, N = 181) = 12.2, p < 0.01, η2 = 0.07; 3 Kruskal-Wallis H-test = (3, N = 180) = 11.9, p <0.01, η2 = 0.07; 4 Kruskal-Wallis H-test = (3, N = 184) = 20.5, p < 0.001, η2 = 0.11; 5 Kruskal-Wallis H-test = (3, N = 182) = 24.8, p < 0.001, η2 = 0.14; 6 Kruskal-Wallis H-test = (3, N = 181) = 16.5, p < 0.01, η2 = 0.09; 7 Kruskal-Wallis H-test = (3, N = 179) = 12.8, p < 0.01, η2 = 0.07; 8 Kruskal-Wallis H-test = (3, N = 182) = 30.3, p < 0.001, η2 = 0.17; 9 Kruskal-Wallis H-test = (3, N = 182) = 15.9, p < 0.01, η2 = 0.09; 10 Kruskal-Wallis H-test = (3, N = 179) = 15.4, p < 0.01, η2 = 0.09; 11 Kruskal-Wallis H-test = (3, N = 179) = 18.5, p < 0.01, η2 = 0.1.

Share and Cite

MDPI and ACS Style

Rojo Robas, V.; Madariaga, J.M.; Villarroel, J.D. Secondary Education Students’ Beliefs about Mathematics and Their Repercussions on Motivation. Mathematics 2020, 8, 368. https://doi.org/10.3390/math8030368

AMA Style

Rojo Robas V, Madariaga JM, Villarroel JD. Secondary Education Students’ Beliefs about Mathematics and Their Repercussions on Motivation. Mathematics. 2020; 8(3):368. https://doi.org/10.3390/math8030368

Chicago/Turabian Style

Rojo Robas, Vanesa, José María Madariaga, and José Domingo Villarroel. 2020. "Secondary Education Students’ Beliefs about Mathematics and Their Repercussions on Motivation" Mathematics 8, no. 3: 368. https://doi.org/10.3390/math8030368

APA Style

Rojo Robas, V., Madariaga, J. M., & Villarroel, J. D. (2020). Secondary Education Students’ Beliefs about Mathematics and Their Repercussions on Motivation. Mathematics, 8(3), 368. https://doi.org/10.3390/math8030368

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop