SROI Methodology for Public Administration Decisions about Financing with Social Criteria. A Case Study

Abstract

The measurement of impacts has been considered one of the best methodologies to evaluate the level of achievement of social entities’ objectives as well as of their contribution to resolving social problems. Those methodologies can guide public policies and subsidies granting, as they help to identify the organizations producing a higher social value, and the effects of their projects. Our research focused on the effectiveness and the efficiency of social entities, measured through their capacity to generate impacts on their stakeholders. The research was realized through the analysis of a case study: the special education center for disabled youths, CEE-SA, in Spain. The social return on investment (SROI) methodology has allowed us to monetarize the social value created for stakeholders through the activity carried out by CEE-SA, and it provides information about the whole value creation process that is generated, for which the analysis and follow-up through the indicators offers a contribution to its management system. This case study can serve as a reference in assessing the management processes of similar entities and can also highlight SROI usefulness for public administrations as an assessment tool for subsidies granted on social criteria. The originality of this research relies on the new SROI methodology provided for the assessment of public financing decisions, especially in a field that remains as under-researched as special education schools.

1. Introduction

Public policies and services’ efficiency and effectiveness have been analyzed for years throughout the results of projects and budget fulfilment. However, citizens have become more and more concerned by the usage of public funds in a context of expenses containment and a higher tax pressure. In this context, the public sector needs social value measurement and assessment tools [1] in order to increase citizens’ trust and gain legitimacy. On one hand, trust is promoted through transparency and showing the guarantee of a service’s sustainability. On the other hand, legitimacy increases as public entities satisfy their stakeholders’ expectations, sometimes delegating a service’s provision in social entities financed with public funds.

In the current environment, in which social challenges such as dependency, handicaps, or inequality are demanding a huge quantity of public resources, the analysis of public policies’ efficiency and effectiveness becomes increasingly meaningful. Frequently, the public sector delegates attention to these issues to social organizations working to give solutions to the most vulnerable groups [2]. In order to justify public subsidies to social entities, work is frequently done to relate the cost of the interventions with their results [2]. However, these investigations do not fully reflect the social value generated by these organizations, since the analysis is dominated by an eminently economic approach, despite the fact that many nonprofits prioritize social objectives of an intangible nature over monetary objectives [3,4,5]. In addition, all the stakeholders involved are not always included in the analysis, so the effects studied are limited to the direct beneficiaries [2].

Recently, the debate on the efficiency of nonprofit organizations (NPO) has focused on measuring the impacts on the groups that are affected by their activities [6,7]. The measurement of impacts has been considered one of the best methodologies when evaluating the level of achievement of social entities’ objectives and of demonstrating their contribution to resolving social problems [8,9,10,11]. Efficiency and effectiveness measurement in these entities shows, somehow, the suitable assignment of the public resources, as public administrations finance their activity. That explains the interest shown by the rulers in the usage of tools based on value analysis, as they combine economical assessments of the subsidies’ projects with the social impact generated, helping in this way to legitimize the granted subsidies in the eyes of the public [12].

In the education sector and, in particular, organizations dedicated to training young people with intellectual disabilities, impact studies are rare [13]. From a public service’s sustainability point of view, this approach is interesting as these entities receive a relevant percentage of public finance, and they produce a significant impact on crucial aspects related to vulnerable groups integration and equality of opportunities.

This requires a change in direction from an approach focused on evaluating the programs and services provided to the beneficiaries individually to the analysis of the changes generated by these entities in the social structures and in the community dynamics, where the beneficiaries are the participants in the entity’s activities [14].

The purpose of our study was to use the SROI (social return on investment) methodology to demonstrate, with basis in scientific evidence, the utility of measuring social impact, both for public funding evaluation and for the internal management of social entities. SROI monetizes the impacts that are not considered in the financial reports or in the evaluations of the programs of the social entities [13,15,16]. First, the information obtained throughout the application of this methodology will provide better scientific evidence to public administrations to evaluate the changes and impacts that their activities produce, in order to prioritize the resources they assign [1]. At the same time, citizens will be able to evaluate the social value obtained with the taxes they pay, as it is measured in money. Moreover, justification of the subsidies that are granted should be related to people’s wellbeing changes, which are crucial in their mission fulfilment as public entities. Secondly, the identification and quantification of the impacts provides new scientific evidence for the management of social entities, through a clearer visualization of the changes caused by their managerial decisions.

To achieve these purposes, our contribution relies on a case study that analyses the social value contributed to stakeholders by the activities carried out by a social entity that provides educational services to young people with intellectual disabilities (CEE-SA) in Spain. The selection of this case study is justified because CEE-SA works in a sector of high social relevance, as disability is. Moreover, despite the importance given by the public to this sector, research addressed at understanding the social value generated as SROI has not been applied so far. For this reason, our research can provide new scientific and practice implications. This case analysis can serve as an example of good practices in analyzing the social value created, and as a reference in assessing the management processes of similar entities. Moody et al. [15] also emphasized that despite the interest in SROI, case studies are still needed to analyze its implementation and to highlight its potential benefits.

This research contributes to deepening the knowledge of SROI methodology as an adequate scientific tool to understand and improve the effectiveness and efficiency of public administration funding and social entities management. Specifically, this study provides news that may be relevant in the internal management of special education schools, while also contributing to the generation of knowledge to justify the funding granted by the public sector based on the analysis of the social value of a NPO.

The originality of our case study relies on the application of the SROI methodology to evaluate public financing decisions. The application of SROI offers elements that can help public administration to decide about financing with social criteria, according to the social impacts generated by each kind of activity. In addition, even though scientific literature on SROI is a growing body of knowledge, there is still no application in the field of special education to young people with disabilities, which is a social challenge of increasing interest to the public.

The remainder of this article is organized as follows. Section 2 presents a review of the relevant literature on social value and highlights the importance of this concept for social entities. Section 3 presents our application of SROI methodology. Section 4 delves into the empirical research as well as the study results. Lastly, Section 5 includes a discussion of the study results, conclusions, limitations and further research.

2. The Measurement of Social Value in the Nonprofit Sector

Nonprofit organizations combine their social goals with the search for greater effectiveness and efficiency, which guarantee their ability to continue providing services [17]; therefore, evaluating their performance in developing their activities is gaining greater importance [10,18]. From the resources and capacities theory point of view [19], the performance analysis should allow researchers to demonstrate which entities are applying the funds received in the best possible way, and consequently, are more competitive, as in the current environment, characterized by expenses containment, public finance access has become more restricted.

Given the growing presence of social entities in the economy and their essential role in maintaining a state of well-being, the pressure to demonstrate their contribution to solving social problems has increased on the part of policymakers, resource providers, donors, users, volunteers, and other third parties [10,17,20,21]. In particular, the funders want to know if their funds are being well spent and are contributing to achieving the purposes for which they were contributed [16]. This concern has led to the development of initiatives for measuring social value since the 1990s [10].

The measurement and management of performance in social entities has two main aims: to demonstrate its contribution to value creation and to improve organizational performance [8,10,18].

However, these tasks are not without difficulties. With regard to the first aim, verifying the social value created, the first difficulty is associated with the fact that these entities work towards achieving social objectives that are intangible and, therefore, difficult to measure [22]. On the other hand, from the point of view of stakeholder theory [23], methodologies for measuring social value that respond to the expectations of a broad spectrum of groups and stakeholders are needed [20]. By only attending to their interests, these entities will get the legitimacy needed to operate. From the legitimacy theory [24] point of view, social impact evidence can support the granting of public funds, as the generated changes are clarified, improving transparency, confidence, and credibility.

Nevertheless, the lack of consistency in providing evidence on the social value created is another one of the difficulties added to the evaluation problem [7,21], since the indicators used can be highly subjective because they cover intangible aspects.

In this sense, the so-called participatory action methodologies [25] could contribute consistency to the results, as they are based on the evidence provided by those affected. This means that it is necessary to involve the stakeholders and rely on their collaboration [26], highlighting the importance of the dialogue and the relevance of the story they provide regarding the credibility of the impacts to be measured [4,9].

Regarding the second aim, improving performance, according to Costa and Pesci [17], each of the stakeholders evaluates the effectiveness of the organization in a different way according to their own perceptions, judgements, and priorities regarding what is relevant, which makes measuring performance difficult. Lee and Fisher [27] found that measures of performance remain a challenge for these organizations, particularly when the expected impacts on beneficiaries are influenced by external factors that are beyond the control of the entity and that question the attribution of the results to a particular intervention.

All these difficulties have generated both reluctance and enthusiasm with respect to some of the methodologies for measuring social value that have emerged recently [21] and that replace the traditional cost-effectiveness and cost–benefit analysis tools used so far [28]. Owen et al. [13] focused their criticism on the fact that these evaluations have been carried out in terms of economic viability and not of social impact.

These limitations are easier to overcome when performance measurement systems use methodologies based on measuring impacts and not on intervention outcomes [7,29,30]. These are mixed methodologies that combine the quantification of impacts with a narrative and dialogue with stakeholders [17,26].

Ebrahim and Rangan [31] defined the impacts as the long-term results derived from an organization’s activities, which produce economic, social, or environmental changes. The concept of impact is derived from the theory of change [32] and from the methodologies based on the logic models backed by some international governmental development aid agencies. These models relate inputs, activities, outputs, and outcomes. However, while the inputs, activities, and outputs depend on the efficacy and internal efficiency of the organization, the outcomes are highly influenced by environmental conditions and stakeholders’ perceptions [1,10].

One of the tools for measuring social value, the use of which has grown the most in recent years, is the SROI methodology [5,11].

Several reasons justify the usage of SROI to focus the public administration’s decisions. In the end, this methodology can help the public sector (and/or the social investors) to evaluate and compare the investment options by the analysis of the changes obtained; it also can facilitate an input–outcomes analysis. Moreover, the impact description made by the stakeholders shows their perception about risks and externalities associated with social interventions. That information helps the administrators to better understand how these organizations contribute to the fulfilment of their goals. The monetarization of impacts improves the public value accounting and the public’s understanding of policies becomes more visible [2]. Finally, this methodology can help managers of social organizations to evaluate and maximize social benefits; moreover, it can be a tool to compare the performance of different managers, allowing them to make more justifiable decisions.

According to Cooney [4], expanding on this methodology will help reduce the criticism of the proposed SROI analysis being an exercise in "rhetoric" and ensure that the lack of consistent results is remedied. This study also responds to the need expressed by Jones et al. [33] to advance the methodology and to share experiences that allow knowledge generation in this field.

3. Methodology

The scope of this study is a case analysis of the special education college CEE-SA, which is a center that receives public funding from the Education Department and is promoted and sponsored by a private foundation that also provides financial resources. CEE-SA serves students with special educational needs due to intellectual, physical, and sensory disabilities, aged between 3 and 21 years.

The pedagogical structure of the CEE-SA is organized around the following two types of teaching:

Basic Compulsory Training (Formación Básica Obligatoria, FBO), for boys and girls from 3 to 18 years old, with seven approved and subsidized units that are distributed as follows: four autistic units, one psychiatric unit, one multipurpose unit, and one hearing and language unit.

Program for the Transition to Adult and Working Life (Programa para la Transición a la Vida Adulta y Laboral, PTVAL), aimed at young people from 18 to 21 years of age, which has three approved and subsidized units, which include two autistic units and one psychiatric unit.

The center offers speech therapy, counselling, medical and rehabilitation services, programs for developing personal autonomy (pilot), swimming, and asinotherapy, among others. In addition, it provides school transportation, dining services, and morning classes.

As already mentioned, one of the most innovative methodologies in the field of measuring social value is the SROI methodology. In this case, it was applied to an evaluative analysis [34] about a real situation involving an activity developed during an academic course (2016–2017), which allowed for the identification of the priority stakeholders, analysis of the main changes generated, definition of a set of indicators that allow the monitoring of the changes, and, finally, the monetary quantification of the impact or social value contributed to the stakeholders.

Several reasons make the CEE-SA a significant case in order to analyze how public administrations can use impact assessment to direct their funding decisions and policies. Firstly, it is an institution for which feasibility is highly conditioned by public funding, as it provides a deficit service due to the huge quantity of activities and attention demanded by the beneficiaries, and is impossible to finance with the family’s contributions. On the other hand, it is a not financially sustainable institution through the beneficiary’s contributions, as the action is developed in an especially vulnerable environment, as the school is located in one of the most depressed neighborhoods of the city—that is why its sustainability can be measured only in social terms, using the measure of intangible impacts.

Secondly, the school deals with a problem that constitutes one of the big challenges in our environment: disability attention, which implies a wide group of people demanding a huge quantity of public resources. The significance of this group and the funds granted, give better opportunities to improve the efficient assignment of public resources—that is the reason why the case offers a higher learning effect.

Finally, the theory of change of CEE-SA is based on an intervention model developed in a special education school, opposite to other integrated education models. The case contributes to the debate on educational models for disabled youth, analyzing the impacts generated in this differentiated educational environment.

The information was obtained through interviews and questionnaires to the priority stakeholders (the university’s ethics committee has evaluated the project in accordance to the criteria established by the rules for the protection of people and the respect of human rights in scientific activities): the FBO students (30 students), the PTVAL students (12 students), the families (42 families), the center staff (15 subsidized employees and 10 contracted employees by the center), volunteers (11 people), the Education Department administration, public administrations, and other institutions (university, collaborating companies, city council, etc.).

The impacts were identified through a system of indicators. To obtain the data related to these indicators, 98 surveys were distributed among the different interest groups. At the time that the information about the impacts on students was gathered, the difficulty arose that the students are children under 18 years of age, and all of them have some type of intellectual disability. Therefore, the information related to these beneficiaries was collected from the students’ teachers and families. Two surveys were administered for each student, as follows: one completed by the staff of the center and another by the closest relative.

4. Results

The results obtained in analyzing the social impact of the CEE-SA correspond to the application of the different phases of the SROI methodology [34].

4.1. Stakeholder Identification

The quantification and monetization of the impacts that the CEE-SA generates in society requires the identification of its main stakeholders, as the process must be highly participatory, so that the changes that the school’s activity generates in them can be evaluated. According to MacIndoe and Barnman [6], social entities should identify two types of stakeholders: direct and indirect beneficiaries of the service provided, among which we highlight students and their families, and institutional stakeholders, who can influence the decisions from which the generated impacts depend. The latter stakeholder group can consist of the following three types: resource providers, internal staff, and collaboration networks.

Following the principles of the SROI methodology, the stakeholder identification phase is an ideal time to favor the involvement and participation of stakeholders throughout the process [15,35]. For this reason, different focus groups were organized with the representatives of the foundation, the members of the school’s administration team, and the president of the Association of Parents of Students (Asociación de Padres y Madres de Alumnos, AMPA). In the focus groups, the methodology was clearly explained and a brainstorming session was carried out to identify the priority interest groups.

After the analysis and consensus among the focus group participants, it was decided to focus the study on the following seven priority groups: the FBO students, the PTVAL students, the families, the center staff, volunteers, the Education Department administration, public administrations, and other institutions (university, collaborating companies, city council, etc.).

4.2. Identification of Outcomes by Stakeholders

Then, different meetings were held with different representatives of the interest groups to determine the core changes that the activity of the school generates. While these interviews encouraged the stakeholders to speak spontaneously about how they are involved in the center’s activity and how they perceive the impacts they receive [36], their participation in the process also provided greater consistency and robustness to the results and counteracted subjectivity in some measurements that are necessary for SROI application [5,35].

Following the type of analysis previously carried out by Mook et al. [16], the following three types of impacts were considered: economic, socioeconomic, and social, since it is sometimes difficult to separate some impacts from others. Likewise, the categories suggested in the study by Farr and Cressey [2] on the impacts of disability care programs were considered, which were as follows: individual impacts and impacts on interpersonal relationships, on institutional relationships, and on policies related to well-being.

Different authors suggested that given the restrictions on resources available to make the SROI system operational, it is necessary to limit the number of identified impacts to the most relevant ones [5,15,16]. Out of the set of possible outcomes, after excluding changes that lack materiality, either due to the lack of information, due to the difficulty in quantifying them, or because it is a minor change, a final relationship was established consisting of 31 outcomes (

Table 1. Parameters for impact quantification.

	Changes	Indicators	Proxy
Students	Acquisition of basic skills	Level of perceived change	Hourly cost for a social educator
	Acquisition of psychomotor skills		Hourly cost for a physiotherapist
	Acquisition of learning habits		Hourly cost for a psychopedagogy practitioner
	Development of cognitive and communication skills
	Improvement in the self-regulation of feelings or emotions		Hourly cost for a psychologist
	Acquisition of social skills		Cost for a 10-day special education camp each year
	Acquisition of autonomy habits		Hourly cost of a special education technical assistant
	Incorporation in social life and working life		Hourly cost of an employment counsellor
	Acquisition of pre-employment skills		Cost of a 10-hour workshop
	Contact with the working world		Cost of a visit
Families	Knowledge of resources and network support	Level of perceived change	Hourly cost of a social worker
	Free time (FBO) (before school care)	Number of morning classroom hours	Hourly cost for a free time monitor
	Free time (FBO and PTVAL) (extracurricular activities)	Number of hours of extracurricular activities
	Free time (FBO-PTVAL) (compulsory training)	Number of hours in the center	Daily cost for a daytime stay
	Improvement of parenting skills	Level of perceived change	Hourly cost for a psychopedagogy practitioner
	Family respite (camp activities organized by the AMPA)	Number of days that families enjoyed free time because their children participated in the activity	Daily cost for camp for disabled children
	Guidance, more security and calm to face special education	Level of perceived change	Hourly cost for a psychologist
	Decrease in family problems		Hourly cost for a family mediator
Staff	Identify or sense of belonging	Level of perceived change	Average of the annual salary difference to change jobs
	Job creation	Nº of employees	Average annual cost
	Training and knowledge in special education	Percentage of employees who report having improved their knowledge	Cost of a professional refresher course
	Value loss due to temporary hiring (-)	Number of hours of overworking until the person–position adjustment is achieved again	Hourly cost according to the category of the person affected
Volunteers	Change in the conception of special education	Level of perceived change	Average course cost
	Social commitment with other interest groups		Cost of a similar activity
	Personal growth, development of values	Number of volunteering hours	Adjusted hourly cost of an education technical assistant
Department of Education	Improvement of the performance in the Junta de Andalucía in matters of special education	Number of good practices identified	Consultancy contract for procedures in special education (270 hours)
Public Administrations	Increase in income from social contributions	Number of employees who contribute to social security	Annual average social security contributions
	Increase in income from taxes (personal income tax—PIT)	Number of employees who pay PIT	Annual valuation according to the average PIT rate
	Indirect savings in unemployment benefits	Number of people who do not earn unemployment benefits	Average annual value of unemployment benefit
Other institutions	Change in the conception of special education	Number of institutions with which they are related	Not applicable
	Personal growth, development of values	Number of actions carried out with these institutions

).

4.3. Quantification of Outcomes by Stakeholders

The process of quantifying outcomes requires identifying the indicators that can determine the occurrence of changes in the most objective possible way. In accordance with Slater and Aiken [12], the development of standardized indicators systems for nonprofit entities is highly important. Nevertheless, linking indicator to impacts may generate a performance measurement system based on stakeholder’s perceptions and in a deliberative process that confers the system a higher consistency.

For quantifying the impact, the indicator results were weighted by the number of people affected by the change, thus determining its outcome incidence [36].

Once the outcome incidence is quantified, it is necessary to assign a monetary value that estimates the value created. This monetary value estimation allows an approach to the economic discourse and the performance analysis. The choice of proxies is one of the most involved phases of the SROI methodology [11]. For this step, it was necessary to resort to secondary sources of information and to the subsequent comparison of the chosen proxies with the CEE-SA management team. The sources used by other studies were consulted [5], and some of them were adapted to the environment in question, for example, the salary costs of the XIV General Collective Agreement of Centers and Services for people with Disabilities have been used as proxies in the gaining of certain skills, abilities, and habits. To avoid overestimating the impacts, the most conservative assessment was always chosen.

Table 1 includes the indicators that measure the occurrence of the change. As seen, the indicators indicating the level of change perceived by stakeholders predominate, together with activity measurement indicators. In addition, the proxies chosen for the monetization of the impacts are summarized.

To ensure that the impacts generated by the organization are not overestimated, once the changes are assigned a value, the SROI methodology requires the following two adjustment coefficients to be applied: deadweight and attribution.

Deadweight considers the amount of impact that would have been generated in any case, even if the entity providing the service did not exist. In practice, this percentage is the result of a counterfactual analysis [3,28].

Attribution acknowledges that a portion of the changes could have been generated by the intervention of other actors, since it is very common to achieve social objectives through the intervention of organizations that work in a network in the same field and that complement each other.

According to Pathak and Dattani [3], in the absence of objective data on these coefficients, it is best to use estimates made internally by agents familiar with the project or intervention environment. According to Ryan and Lyne [37], to give robustness to this type of measurement, it is necessary to involve stakeholders in the SROI process. For these reasons, and in the absence of previous studies and statistics that allow us to determine which portion of the impacts cannot be interpreted as belonging to the school, the adjustment coefficients have been estimated through surveys to stakeholders, in which the stakeholder stated what percentage of the impact they consider should not be attributed to the CEE-SA intervention. Specifically, they were asked to valuate, from 0 (disagree) to 5 (totally agree), the possibility of obtaining the change if CEE-SA did not exist, or if the change could be generated by any other organization.

Table 2. Summary of the impact assessment of CEE-SA.

	Change	Outcome Incidence	Attribution	Deadweight	SROI Impact	Subtotal
FBO students	Acquisition of basic skills	25.25	46.25%	50.42%	€12,972.68
	Acquisition of psychomotor skills	24.13	47.5%	49.17%	€9929.32
	Acquisition of learning habits	24.50	39.17%	42.08%	€9037.29
	Development of cognitive and communication skills	24.00	40%	38.75%	€9234.12
	Improvement in the self-regulation of feelings or emotions	24.38	38.75%	38.75%	€9573.79
	Acquisition of social skills	24.00	37.92%	38.75%	€4517.49
				Total student impact (FBO)		€55,264.7
PTVAL students	Acquisition of basic skills	8.18	52.27%	54.55%	€2053.05
	Acquisition of psychomotor skills	7.50	51.14%	53.41%	€2633.25
	Acquisition of learning habits	7.77	46.59%	47.73%	€2271.91
	Development of cognitive and communication skills	7.50	45.45%	47.73%	€2238.84
	Improvement in the self-regulation of feelings or emotions	6.95	46.59%	47.73%	€938.2
	Acquisition of autonomy habits	8.45	45.45%	46.59%	€2445.56
	Incorporation in social life and work	5.45	40.91%	40.91%	€570.51
	Acquisition of pre-employment skills (including the concept of company)	6.41	40.91%	40.91%	€3316.55
	Contact with the working world	5.55	40.91%	39.77%	€612.31
				Total student impact (PTVAL)		€17,080.16
Families	Knowledge of resources and the support network	24.09	41.18%	76.47%	€603.32
	Free time (FBO) (before school care)	3206.25	0%	61.9%	€18,321.43
	Free time (FBO and PTVAL) (extracurricular activities)	684	0%	61.9%	€3908.57
	Free time (FBO) (compulsory training hours)	31,350	0%	100%	€0
	Free time (PTVAL) (compulsory training hours)	8621.25	0%	43.75%	€32,833.95
	Improvement of parenting skills and raising children	25.94	39.71%	76.47%	€971.58
	Family respite (activity camps organized by the AMPA)	30	0%	0%	€1485
	Guidance, more security and calm to face special education	31.5	26.47%	55%	€5503.24
	Decrease in family problems	24.09	43.75%	64.58%	€4146.19
				Total Family impact		€67,773.27
Staff	Identity or sense of belonging	23.44	0%	38.75%	€6316.41
	Job creation (subsidized employees)	15	0%	88.8%	€48,847.46
	Job creation (contracted employees)	10	0%	88.8%	€23,775.41
	Training and knowledge in special education	20.31	50%	61.54%	€859.38
	Value loss due to temporary hiring (-)	30	0%	0%	€−637.29
				Total Staff impact		€79,161.36
Volunteers	Change in the conception of special education	82.64%	0%	0%	€33.88
	Social commitment with other interest groups	74.59%	0%	21.21%	€26.44
	Personal growth, development of values	449.09	0%	9.3%	€777.77
				Total Volunteer impact		€838.1
Department of Education	Improvement in the performance of the Junta de Andalucía in matters of Special Education	1	0%	0%	€29.700
				Total Department of Education impact		€29,700.00
Public administrations	Increase in income from social contributions	2.8	0%	88.8%	€26,144.23
	Increase in income from taxes (PIT)	2.8	0%	88.8%	€12,157.07
	Indirect savings in unemployment benefits	2.8	0%	88.8%	€30,858.24
				Total Public Administration impact		€70,445.3
				TOTAL		€320,262.9

includes the occurrence of changes, attributions, and corresponding deadweights, as well as the final impact created by the school for each type of stakeholder.

Although the methodology indicates the importance of analyzing the duration of the change [34], this study was carried out for an economic year, which coincides with the academic year and therefore with the valuation of the financial inputs of that year. In this case, being cautious in the assessment has been considered more opportune, since it is the first time that this methodology has been applied in these types of centers and because the study is about changes in the disabled and there is no certainty that these changes will last over time [38]. In turn, this limitation is evidence that the study has not overestimated the results.

Regarding the FBO student body, the changes with the greatest impact were those corresponding to the acquisition of basic skills (€12,972.68) and other skills whose valuation exceeded €9000. To a lesser extent was the contribution to the improvement of social skills (€4517.49), whose assessment was affected mainly by the lower availability of activities that offer this training. Taking into account that these changes affected a group of 30 children, we can deduce that the impact generated for each student amounted to €1842.16.

The changes in the acquisition of skills and abilities that were also present in the PTVAL students had similar results to those of the FBO students, although with a lower occurrence. To these changes were added other specific changes to facilitate the integration of PTVAL students into the world around them, and thus strengthen their autonomy. In fact, the acquisition of skills for insertion into the workforce showed the highest impact in this group of young people (€3316.55). In this case, each one of the 12 students of the program received an impact valued at €1423.35.

The total assessment of the changes that the school generated in the families of the students resulted in a value of €67,773.27. This is the group that most benefited from the change, with an impact per family of €1613.65. Although families did not take full advantage of the resources provided by the school, or despite the availability of other alternatives (as seen in the attribution and deadweight parameters), it was one of the most notable impacts among the different interest groups.

The impact of personnel was measured mainly in economic terms, by the contribution of the school to job creation. As seen in Table 2, the total amount of generated changes was €79,161.36. It should be noted that this impact included a change in social nature, which is related to the greater satisfaction felt by the employees of this type of entity due to their identification with the social purposes of the center (€6316.41). Likewise, it is worth highlighting not only the positive changes but also the negative changes, which is why we have identified how the decisions related to temporary contracts result in a negative impact, although of a smaller amount (€637.29), but whose identification provides very valuable information for the decision making of the entity’s management.

Regarding the volunteers, the CEE-SA and a neighboring school maintain a collaboration agreement whereby the students of the latter develop volunteer activities. Specifically, volunteers come to the CEE-SA on Fridays to share activities with the FBO and PTVAL students, initiating certain changes in, for example, developing values, such as equality, or the knowledge of new realities. The economic valuation of this relationship was quantified at €838.1.

The impact on the Department of Education was attributed to the recognition by the Andalucía School Council as an example of good educational practices in the field by the Transition to Adult and the Working Life Program (PTVAL). In this way, the school offers a practical and effective educational response for students to work on curricular aspects and, in turn, develop the necessary skills for later job placement. This good practice, which the Department of Education can transmit and implement in other special education centers of Andalusia, has been valued at €29,700.

Finally, regarding the stakeholder public administrations, job creation also causes a direct economic impact, either as a consequence of the contribution made by these workers to the social security contribution system, to the tax system for the personal income tax, or as a consequence of the saving that occurs in unemployment benefits. The sum of the three impacts considered amounted to €70,445.30.

4.4. SROI Rate Calculation

The next step of the SROI methodology involves calculating the ratio that relates the monetarized impacts with the available resources.

In this case, the financing entity provides resources that allow the sustainability of the school activity, which was valued at €106,261.56 in the academic year 2016–17.

Following the proposal of Millar and Hall [39], different SROI rates were calculated with the following two types of numerators: considering only the social impacts, which are qualitative as they have been previously monetarized through the allocation of a proxy; or also adding the economic impacts that are in themselves of a monetary nature.

First, SROI taking into account both social and economic changes presented a result of €3.01. This means that for each euro invested, €3.01 was returned to the different interest groups. On the other hand, if we only analyzed the strictly social impacts—that is, if we excluded the changes related to job creation, which affect both the staff and public administration stakeholders—the resulting SROI would be €1.68.

The SROI rate is much more than a number, since it is accompanied by the changes induced by the organization that support numbers [3,35]. What is truly important is the process followed until the SROI rate is obtained [4,15], which has allowed evidence about the sources of added value and the impacts generated by the CEE-SA school in its environment to be obtained.

Once the SROI has been calculated, it is necessary to recognize that, to quantify the variables and monetize them, subjective criteria have been used, resulting in some uncertainty about the reliability of the results obtained. The SROI methodology attempts to reduce this uncertainty by proposing different scenarios [5,15,16]. In this case, a sensitivity analysis was carried out for those changes that had a higher occurrence (excluding the job creation and its indirect effects), which consisted of determining the possible effect of an increase or decrease in the proxy by 10%. The results showed that the effect was practically insignificant, changing slightly from €3.01 to €3.102 or €2.926, respectively, which supports and strengthens the results of the valuation of the changes and values of the SROI previously presented.

5. Discussion and Conclusions

This research contributes to understand how the SROI methodology can help to elucidate the social impact generated by NPO activities, thus responding to one of the management challenges of social entities for which economic performance is not the main objective [4,5]. The application of the SROI methodology to the CEE-SA case study has not only allowed the impacts generated by the activity carried out for the different stakeholders to be quantified, but also demonstrated the whole value creation process that is generated. The monetarization of these impacts will allow to CEE-SA prioritize funding decisions according to the importance of the changes achieved, as well as promoting strategies more coherent with the social mission of these entities. Further, the methodology has allowed us to create a scorecard to support the process of decision-makings, for which analysis and follow-up through the impact indicators are contributions to the management system of these organizations.

Our analysis also contributes to provide orientation on the financing of public policies to NPOs, as these policies must guarantee the sustainability of those services facing social problems, such as the one experienced by CEE-SA. Quantifying social value in financial terms makes it possible to improve the understanding of impact by using a common language and by making it possible to visualize its importance [3], that is why this analysis allows us to evaluate the contribution to social value generation in those organizations receiving public funds; this analysis is consistent with the need to take into account social criteria to focus the activity of the public sector, in accordance with international assessments as that of the Directive 2014/24/EU, of February 26th, promoting social clauses in public contracts.

In accordance with Farr and Cressey [2], one of the main advantages of this instrument is that the parameters have been estimated in a participatory manner, which has shown the impact generated in students, families, and society in general; through the participants of the community with which it relates; and other entities with which best practices are shared, such as, in this case, the impact on the Department of Education. Stakeholders’ opinions about the changes they perceive thanks to the school’s actions and the SROI analysis performed led us to make some recommendations influencing the efficiency and effectiveness of the organization.

Firstly, regarding the impacts generated in the families, there are no similar previous studies with which the results can be compared, since the analysis requires that the study participants’ conditions be similar to each other. However, the data obtained showed that the family is a stakeholder that is affected greatly by the activities developed by the school [40] and benefits in accordance with the extent that it identifies and collaborates with the entity. This methodology application and the disclosure of the information obtained should be a way to encourage citizenship participation in public policy development. In this case, the low level of integration and collaboration between families and the school is worth noting, which is evident in evaluating the indicators and the correction coefficients and is perceived in the recognition by stakeholders that these changes are partially attributable to the school’s activity; therefore, the impact was lower than what could be expected. In any case, the results have shown the need for the center’s management team to strengthen relations with families and increase communication mechanisms and transparency with them.

In this sense, it should also be noted that the results of analyzing the changes in the PTVAL students revealed that, although this program has been chosen as an example of good practices by the Department of Education, the families and school staff recognize that the changes achieved and generated by the students are not as high as desired. This fact may result from the expectations placed on these groups not fully corresponding to reality, due primarily to the variability in the profiles of the students who access this program. All this leads us to prudently suggest a strategic review of the Transition to Adult and Working Life Program. One of the weaknesses that the previous analysis can present and that may be influencing the result of this change is that the assessment made regarding the PTVAL students has not taken into account the particular circumstances of each one of them. An improvement to be considered in future studies could be to define personalized objectives for each student according to their degree of disability and their personal situation, so that from that information, an individualized measurement of how they have been able to respond to such changes can be obtained. These results should be reviewed for resource planning and taking actions that allow students to make progress towards the changes.

Regarding the assessment of the impact on students, previous studies both support and contradict that people who are related to the disabled, whether they are employees of the service provider or relatives, can provide valid information to assess their well-being [41,42]. However, in the case of minors, or in the case of people who cannot make decisions for themselves, it is understood that the information obtained represents a multidimensional perspective that allows us to evaluate the change generated in them.

Another of the impacts that has been evidenced and that is key in these types of entities is the identity or sense of belonging of the employees. The degree of occurrence shows that the staff feels satisfied with the activities they perform. The staff do not perceive their work as just another job but rather identify with the mission of the entity and with the consequent impact that this has on the delivery of the activity, which is very necessary in NPOs. This result could be used by the institution to help strengthen the relationships with other stakeholders and facilitate the social objectives achievement.

In quantifying the impacts, it is noteworthy that the most important changes that the school generates in its stakeholders are those related to job creation, even though they were corrected by the employment rate in the service sector, both in the personnel and in public administrations.

In short, in line with the findings of Moody et al. [15], by applying this methodology, the organization learns about how it is responding to the achievement of the expected changes, involving the parties involved. Disclosure of the information obtained in this process can contribute to reviews of public policy suitability as a result of being offered key information to analyze the effectiveness of organizations, and in the end its efficacy.

In short, among the main contributions of this research is that it provides a coherent and clear approach of the activities of a special education school, which allows internal management of the entity based on the analysis of the changes generated by the organization in its main stakeholders, improving its contribution to the well-being of people with disabilities and their families. The mentioned processes could be a reference for this methodology to be adopted by those public or private administrations developing this kind of activity or offering these services.

In addition, this improvement in the understanding of the value sources contributes to inducing changes in the strategy, in the rendering of accounts, in the capacity to manage the risks, identify opportunities, and obtain the necessary financing to achieve its mission by providing evidence regarding the return on the investment, which can justify the use of the received funds [43,44], thus legitimizing the performance of the schools [4,11,22].

With this case study the potential of this methodology is demonstrated in analyzing the impacts produced and, consequently, encourage other institutions to adopt in higher or less depth this tool, as the analysis of the changes over time and their causes will be a management tool that promotes continuous improvement in nonprofit organizations.

However, between the limits of the study, we found that the SROI methodology had limitations, despite its contribution to representing the social value created. First, the social impact is difficult to identify [11,16]. Second, some problems regarding the quantification arise from subjectivity in the scoring and the assumptions required for identifying the different variables needed to apply the methodology [16,33,35]. According to Maier et al. [11], aspects such as the quality and availability of the data, the underlying measurement problems, the causality and the correlation, and the period of time analyzed are limiting factors that should lead SROI users to be cautious when making comparisons between organizations [3,11,16]. Third, if we use SROI to try to justify the financing obtained, sufficient evidence should be provided about the social impact so as not to question the methodology [4,16].

Further research should analyze other organizations with similar projects, using this methodology. That analysis should confirm the kind of impacts produced and would increase the reliability of the proxies found. This way, a response is given to the need for establishing clear and precise measures of the impacts in the social environment [15]. On the other hand, the compared analysis with other entities of the public sector should shed light on the efficiency and effectiveness of this kind of delegation to NPOs.