Validation of the Index for Inclusion Questionnaire for Compulsory Secondary Education Students

Abstract

As one of the protagonists in education, the perspective of the students is fundamental in the determination of inclusive education in an educational center. The Index for Inclusion is an instrument and strategy for self-evaluation. One of their questionnaires, the questionnaire for compulsory secondary education students, is intended for students and has become one of the most used instruments to help teaching teams to self-assess their political and practical cultures from the perspective of the values and principles of educational inclusion worldwide. Some of the questionnaires included in the Index have been used in many studies, mainly in a qualitative way. For this reason, the present study intends to show evidence of validity of the Index for Inclusion questionnaire of students in a quantitative way through an exploratory factor analysis (EFA) and a confirmatory factor analysis (CFA). In this study, 727 secondary school students (359 boys and 368 girls) aged between 13 and 19 years (M = 13.89; SD = 1.35) took part. They belonged to six educational centers in the province of Almeria. To analyze the temporal stability of the Index for Inclusion student questionnaire, a second independent sample of 81 secondary school students was used, aged between 15 and 18 years (M= 16.14; SD = 0.78). The results revealed adequate adjustment rates, showing the invariant structure with respect to gender. The Student Inclusion Index was shown to be a robust and adequate psychometric instrument to assess the degree of development of inclusive education in schools from the perspective of secondary school students, and therefore, its future application to students in schools is recommended.

1. Introduction

Currently, Inclusive Education (IE) is a challenge in the educational systems of all countries [1]. In this sense, Echeita [2] argues that IE allows changing educational systems so that all students, without exception, can fully develop their personality within the framework of a comprehensive and common educational system. In this way, schools are made up of very diverse students, being a true reflection of society today. Responding in the most optimal way to the diversity of students in the educational and social environment has become an inevitable issue of the current educational breviary [3,4]. Undoubtedly, attention to student diversity must be recognized as an indication of quality in education [5]. So, as indicated by Garzón, Calvo, and Ortega [6], the teaching staff that stands as one of the pillars for IE will require a willingness to transform their methodologies and attitudes towards the present educational situation. These methodological and attitude changes will allow them to face the attention to student diversity more successfully. Therefore, we must have instruments that assess the development of cultures, politics, and practices in schools with an inclusive orientation. One of the most used instruments worldwide is the Index for Inclusion.

According to UNESCO [7] recognizes IE as a decisive means to achieve Education for All, highlighting that education is a right associated with each basic and fundamental person of a more equitable and equitable society. Along these lines, Ainscow [8] advocates an education in which the discriminatory processes exhibited in specific behaviors and responses towards diversity are suppressed, caused, among other aspects, by reason of gender, ethnicity, social class, religion, sexual orientation, and disability. Precisely for this reason, the desire to transform the school environment to improve it is raised, an idea from which the inclusion is proposed as a guarantee of equity and quality [9].

1.1. Justification

Several studies support the positive effect of the use of measuring instruments and self-evaluation processes that are subsequently materialized in improvement exercises that defend the transformation towards the increase of practices in an inclusive sense in the educational centers [10,11,12,13]. On the other hand, it should be noted that there are already instruments about attention to diversity for quality IE that are aimed at teachers, students, and families. In the case of the student body that concerns educational and social practices for inclusion, it is classified in the following educational stages: primary, primary and secondary, and higher education. Bravo [14] created the questionnaire that aimed to find out the perspective of primary and secondary school students in relation to IE practices in their classrooms. Likewise, Rinta et al. [15] established the questionnaire called Social Inclusion Assessment Instrument (SIAI) that aims to recognize the link between music and the feeling of social inclusion, as well as implementing an instrument to measure the social inclusion of primary school children. Ferreira, Vieira, and Vidal [16] designed the instrument called System of Indicators for the support of students with disabilities in the university. The purpose of this instrument is to establish a system of indicators that refer to the attention and support of students with disabilities in the Spanish university environment.

On the other hand, we find instruments for the three groups (teachers, students and, families). Moore, Ainscow, and Fox [17] designed the Manchester Inclusion Standard that assesses the level of inclusion in schools located in disadvantaged areas. Along these lines, Arjona [18] prepared the Questionnaire on attention to diversity and the organization of secondary school, which aims to assess the attention to diversity of students and the necessary steps to be taken to address towards inclusion. Alberta Education [19] designed the Indicators of Inclusive Schools, whose objective is to recognize the values and principles necessary to carry out inclusion in schools through the analysis of the opinions of the school community as a whole. The instruments of Alberta Education [19] and that of Moore, Ainscow, and Fox [17] collect information on attention to diversity and inclusion in the stages of pre-school, primary, and secondary education. On the other hand, the Arjona questionnaire [18] collects information related to secondary education.

Without the slightest gender of doubt, one of the paradigmatic tools in the IE is the Index for Inclusion [20], whose main purpose is to support the educational teams through the self-evaluation of their cultures, politics, and practices in the path to inclusion. The original work was published in the year 2000 and its adaptation to the Spanish environment was carried out in 2002, Booth and Ainscow [21] under the name of “Guide for the evaluation and improvement of inclusive education” by the University Consortium for Education Inclusive (http://www.consorcio-educacion-inclusiva.es/). The next English version is from 2011 and the latter was translated into Spanish in 2015 [22]. From the Spanish version from 2015, the student questionnaire we have validated is extracted. The Index for Inclusion has been translated into more than 30 languages [23] and has been used in schools in many countries, including Spain, Durán et al. [24] confirmed their employment in centers, both public and private, in the Community of Madrid, Catalonia and the Basque Country. In the same way, it was used in various educational centers in the Region of Murcia in a research project funded by the Office of Education, Science and Research of the Region of Murcia [25].

This renowned index establishes both a compilation of indicators to promote inclusion in educational centers, as a mechanism to enable self-evaluation work to be provided to schools and a guide for the improvement of IE policies and practices. Note that the questionnaire for secondary school students that we present for validation has been used in a qualitative way in the Index for Inclusion of Booth and Ainscow [26]. The use that has been given to these questionnaires from a qualitative point of view has been very useful. McMaster [27], the educational agents, through processes of reflection, analyzed their personal beliefs and expectations, re-evaluated the values of the school culture, and reinterpreted the educational trend that they are carrying out. A fundamental aspect of improving inclusion in schools is based on understanding the nature of change and giving time to reflect on beliefs that may be deeply rooted. Pillay et al. [28] concluded that it is necessary to raise awareness so that the educational context integrates disability. It is important to involve the entire community in this process. Infrastructure must be adapted and institutional challenges must be overcome. Inclusive education also must be promoted with the presence of all those involved (e.g., teachers, community, family members, and persons with disabilities). Cruz-Ortiz, Pérez-Rodríguez, Jenaro-Rio, Sevilla-Santo, and Cruz-Ortiz [29] demonstrated the relationship between inclusion and quality of life, as perceived by the participants. Neither the presence of SEN or the level of education seemed to influence the quality of life of the participants.

However, we believe that from a quantitative perspective, it can offer relevant information to generate solid evidence that allows for the revision and improvement, where appropriate, of inclusion in the educational centers. The questionnaire that concerns us for the original validation of the Index for Inclusion for secondary school students of Booth and Ainscow [26] has 63 items and three response options: agree, neither agree nor disagree, and disagree.

1.2. Objectives

It is therefore appropriate to see whether through quantitative validation strategies of the Index for Inclusion questionnaire towards students, whose authors are Booth and Ainscow [26], its validity and meaning can be reinforced. Until now, there is no evidence that this instrument has been quantitatively validated despite having been used in multiple studies, which gives it more solidity, more efficiency, more validity, more applicability to the results obtained, and more external validity (more extrapolable to the rest of society). This instrument allows schools to respond satisfactorily to student diversity and overcome barriers to be more inclusive, allowing students to identify their attitude towards inclusion. Likewise, the questionnaire is reinforced since it also allows it to be related to other variables.

2. Method

2.1. Participants

In this study, 727 secondary school students (359 boys and 368 girls) aged between 13 and 19 years (M = 13.89; SD = 1.35 took part, belonging to six educational centers in the province of Almeria. To analyze the temporary stability of the Index for Inclusion for secondary school students, a second independent sample of 81 secondary school students was used, aged between 15 and 18 years (M = 16.14; SD = 0.78), who completed the instrument twice, with a time interval of two weeks between the first and second take.

2.2. Instrument

To measure the attitude of students towards educational inclusion, it was used in Index for Inclusion by Booth and Ainscow [26] translated and adapted by Echeita, Muñoz, Simón, and Sandoval, from the original English version, Booth and Ainscow [20]. The questionnaire is composed of a total of 63 items distributed to a single factor called Attitude towards inclusion. Students responded through a Likert scale from 0 (disagree) to 2 (agree), maintaining the response options of the original scale.

2.3. Procedure

In order for young people to participate, their relatives were required to have a signed informed consent, also informing them of the purpose of the study. Before administering the scale to all participants, it was completed by a small group of people to ensure the correct understanding of all the items. The application of the questionnaire was carried out under the supervision of the principal investigator with extensive experience in the world of research, which explained and resolved the doubts that arose when completing it. The students completed the questionnaire in a well-lit and comfortable classroom at the school that was free of distractions.

The estimated time to complete the questionnaire was around 15 min.

2.4. Data Analysis

To determine the validity and reliability of the Index towards Inclusion, an analysis of its psychometric properties was performed. First, a CFA was carried out to test the factor structure of the questionnaire. Second, multigroup analyzes were carried out to analyze the invariance of the model with respect to gender. Finally, descriptive statistics were provided, and the reliability of the instrument was assessed through internal consistency analysis (Cronbach’s alpha) and a temporal stability analysis (Intraclass Correlation Coeficients (ICC)). For the data analysis, the statistical packages SPSS 25.0 and AMOS 20.0 were used.

Since the Mardia coefficient was high (412.78), for the different CFAs the maximum likelihood estimation method was used together with the bootstrapping procedure. The estimators were not affected by the lack of normality, so they were considered robust [26]. In order to accept or reject the tested models, a set of several adjustment indices was used: χ²/df, CFI (Comparative Fit Index), TLI (Tucker Lewis Index), IFI (Incremental Fit Index), RMSEA (Root Mean Square Error of Approximation) plus its 90% confidence interval (CI), and SRMR (Standardized Root Mean Square Residual). Since χ² is very sensitive to sample size [30], χ²/df was used, considering values below 5 acceptable [31]. Incremental indexes (CFI, TLI and IFI) show a good fit with equal or higher values 0.90 [32], while error rates (RMSEA and SRMR) are considered acceptable with values equal to or less than 0.08 [33,34].

3. Results

3.1. Confirmatory Factor Analysis

Initially, the structure of the 63-item model and a single factor was evaluated, presenting the following adjustment indices: χ² (1890, N = 727) = 7104.97, p = 0.001; χ²/df = 3.76; CFI = 0.68; TLI = 0.68; IFI = 0.68; RMSEA = 0.062 (90% CI = 0.060 - 0.063); SRMR = 0.052. However, after analyzing the covariance standardized residual matrix, the possibility of possible improvements was observed since the residual values of some items correlated with the residual values of other items and were associated with standardized residuals >|2.00| [35]. Thus, a total of nine items were excluded from the model. By eliminating these items, the model fit indices improved: χ² (1377, N = 727) = 5153.07, p = 0.001; χ²/df = 3.74; CFI = 0.78; TLI = 0.78; IFI = 0.78; RMSEA = 0.061 (90% CI = 0.060 -.063); SRMR = 0.053. In this model, it was observed that the standardized residual values of all the items were below two in absolute values and the standardized regression weights were statistically significant (p <.001), ranging between 0.17 and 0.81. Therefore, after observing these data, those items whose regression weights were below 0.5 were eliminated, eliminating a total of 16 items [36]. Excluding these items, the model adjustment rates improved significantly: χ² (665, N = 727) = 2218.21, p = 0.001; χ²/df = 3.34; CFI = 0.95; IFI = 0.95; TLI = 0.95; RMSEA = 0.057 (90% CI = 0.054 - 0.059); SRMR = 0.033. The final resulting model (Figure 1) had standardized residual values of all the items below two in absolute values and the standardized regression weights were statistically significant (p <.001), ranging between 0.67 and 0.81.

3.2. Analysis of Invariance with Respect to Gender

In order to check if the factor structure of the model is invariant with respect to gender, a multigroup analysis was performed.

Table 1. Multigroup analysis of invariance with respect to gender.

Index for Inclusion Model
Models	χ2	df	χ2/df	Δχ2	Δdf	CFI	TLI	IFI	RMSEA (IC 90%)	SRMR
Model 1	3083.87	1330	2.32			0.95	0.95	0.95	0.043	0.038
Model 2	3185.85	1367	2.33	101.38	37	0.94	0.94	0.94	0.043	0.038
Model 3	3185.86	1368	2.33	101.99	38 *	0.94	0.94	0.94	0.043	0.039
Model 4	3288.64	1406	2.34	204.77	76 **	0.93	0.93	0.93	0.043	0.039

* p < 0.05; ** p < 0.001.

shows the various adjustment rates for the four models compared. For the proposed model, no significant differences were found between model 1 (unrestricted model) and model 2 (invariance model in measurement weights). On the other hand, the results did reveal differences between model 1 and model 3 (model of invariant structural covariances) and model 4 (model of invariant measurement residues). The absence of significant differences between model 1 and model 2 constitutes a minimum criterion to accept that the structure of the model is invariant with respect to gender [37].

3.3. Descriptive Statistics, Analysis of Internal Consistency and Temporal Stability

The average scale score was 1.74 while the standard deviation was 0.63. To analyze the reliability of the scale, an internal consistency analysis and a temporal stability analysis were carried out. The internal consistency analysis revealed a Cronbach alpha value of 0.84 for the Index for Inclusion of secondary school students. Regarding the temporal stability analysis, it was performed with an independent sample, analyzing the intraclass correlation coefficients (ICC) and its confidence interval (CI). The results for the Index for Inclusion of secondary school students were as follows: 0.86 (CI = 0.81–0.87).

4. Discussion

Considering that inclusive education implies adopting a socio-ecological approach that takes into account the interactions between students’ capacities and the demands of the environment, stressing that it is the education system that must adapt and respond to all students and not vice versa [38,39,40,41]. Inclusion must go beyond the strictly school environment and take shape as a social and community project. Therefore, inclusion must be worked on “from within” the school community itself, but with the support and sustenance that “from outside”, from the state, autonomous, and local entities must aspire to achieve an inclusive educational system. The Index is an extended tool for guiding and orienting towards inclusion. Its importance is based on reflection, the will to improve, and the research attitude of the schoolchild in the community. Its use has not been limited to the school environment but has been organized in a series of research works that have proven the potential of this resource to guide the decisions that must be made in the implementation of inclusive education. Inclusion is the path and the goal followed by schools in the search for education for all, and research methodologies have been the ways to illuminate this and identify obstacles. Alcaraz and Arnaiz [42] have carried out a longitudinal study on special educational needs in Spain, and the results show that although the Spanish state has made great progress in the commitment to inclusive education, the number of students with SEN enrolled in non-regular schools has increased in recent years. The conclusion is that it is necessary to promote schooling policies for students with SEN which guarantee their presence in regular contexts in order to develop quality and inclusive educational care.

The objective of this study was to show evidence of validity of the Spanish version of one of the questionnaires, referred to students, included in the Index for Inclusion, specifically the questionnaire for secondary school students of Booth and Ainscow [26] translated into the Spanish context by Echeita, Muñoz, Simón, and Sandoval, from the English version of 2011. Until now, this questionnaire had been used qualitatively in numerous studies in order to support, through qualitative self-assessment strategies, inclusion in schools [43,44].

First, the results of the present study revealed through the CFA the support to the factorial structure of the questionnaire made up of 38 items. This result was reached after making multiple adjustments to the questionnaire through two previous CFAs, where a set of items that did not show acceptable adjustment rates were eliminated, since the standardized residual values were greater than >|2.00| so nine items were removed. In addition, after performing the second CFA, it was observed that there were items whose regression weight was less than 0.5, eliminating a total of 16 items. Subsequently, the questionnaire consisting of 38 items had appropriate adjustment indices regarding the analysis of invariance with respect to gender, showing this invariant. In this way, future studies may use the questionnaire to compare scores between students as it is similarly understood by both populations. Subsequently, an analysis of internal consistency and temporal stability of the questionnaire was carried out with a single factor having a value above 0.70 [45].

This instrument (see, Appendix A) can have great value since it allows a better understanding of IE in schools from the perspective of secondary school students, and especially for contributing to the construction of an inclusive society by promoting education inclusive from educational centers. For these reasons, the future use of the Index for Inclusion for secondary school students will be very interesting. Although, on the one hand, the elimination of the 25 items can be a handicap, and on the other, it has a positive side, since, in a smaller way, it allows us to measure the same as with a much broader questionnaire. However, in future studies, the factorial structure of the questionnaire after the elimination of the 25 items should be analyzed, so it is recommended that future researchers analyze said factorial structure in other or similar contexts in order to improve the questionnaire.

5. Limitations

Although the results of the present study show consistent psychometric support, it is necessary to show some of the limitations. This study has been developed only in educational centers in the province of Almeria, so it would be advisable to expand the sample size, extending the research to other territories. Another limitation is the use of self-report measures, which, while appropriate for assessing subjective perceptions, could be supplemented in future research by other types of instruments and informants. Finally, in the present study, 25 items have been eliminated from the initial questionnaire, so it is recommended in future studies to analyze the internal factorial structure with our items and with all the items.

6. Conclusions

As Sánchez-Flores [46] points out, the quantitative approach studies phenomena through rigorous procedures that guarantee precision and objectivity, applying psychometric instruments (typical of the quantitative approach) in order to accredit criteria of rigour. Due to the relevance of this methodology, we have decided to apply it in the validation of our questionnaire. According to the results obtained, we can assert that the present questionnaire has proved to be a robust instrument to assess inclusive education in compulsory secondary education students, showing evidence of validity and reliability.