Using a Multi-Criteria Model to Assess the Suitability of Potential Sites to Implement Off-Grid Solar PV Projects in South America

Abstract

The role of the private sector is essential in providing energy access to all by 2030. However, private investors often face the decision problem of deciding where to implement off-grid projects. This selection process can be considered a multifactorial task because it requires accounting not only for local conditions, technical constraints, and economic-financial feasibility but also for socio-cultural dynamics and environmental consequences. In this paper, we implemented a multi-criteria decision-making (MCDM) method based on the Measuring Attractiveness by a Categorical Based Evaluation Technique (MACBETH) to assist in the selection of the most suitable location to implement off-grid solar PV projects in South America. Following a participatory process with the input of a decision-maker representing the private sector, we combined 13 criteria, within 5 dimensions, to evaluate the performance of three potential sites, namely Bolivia, Peru and Colombia. Based on an overall score associated with each alternative, we grouped them into three categories: highly sustainable, sustainable, and unsustainable. As a result, Colombia represents the highest sustainable option for a potential private investor. This methodology provides guidance not only to investors but also to local utility companies and policymakers assessing off-grid locations with electricity access deficits, in developing countries.

1. Introduction

Rural Electrification (RE), the process of delivering electricity to rural and remote areas, has become a priority in the agenda of developing countries [1]. Globally, rural areas account for 84% of the global population living without access to electricity (640 million unserved people, as of 2019) [2]. In addition, although the number of people worldwide living without electricity has fallen from 1.2 billion (in 2010) to 759 million (in 2019), disparities between rural and urban electrification continue to prevail. Urban areas are approaching universal access, with an electrification rate of about 97%, but in contrast, in rural areas, the electricity access rate is below 80% [1].

Within this context, governments across the world have identified the need of achieving universal electricity access by adopting the United Nations Sustainable Development Goal (SDG) 7, which aims to “ensure access to affordable, reliable, sustainable, and modern energy for all by 2030” [3]. Unfortunately, in some parts of the world, such as in rural and peri-urban areas of Sub-Saharan Africa, the population without access to electricity is growing in line with population growth. This means that the access rate will have to more than triple between now and 2030 to reach universal access. As a result, according to the 2021 energy progress report tracking SDG 7, the world is falling behind to reach this goal and 660 million people will still lack access to electricity by 2030 [2]. In addition, the COVID-19 pandemic has threatened the electrification progress by increasing investors’ risk perception and shifting public funding priorities in developing countries [2]. Thus, as public financing may remain limited over the next few years, it is critical to unlock and promote private financing in the rural electrification sector.

However, off-grid electrification projects are often considered high-risk investments by utilities and off-grid service providers, mainly due to the uncertainty around revenues, project sustainability, scarcity of capital, potentially low returns and the lack of clear and effective public policies [4]. As a result, potential investors and project developers face the decision problem of deciding where to invest in rural and remote areas, while guaranteeing the sustainability of rural electrification projects in the long-term view [5], namely the “capability of the project to be continually operational over its useful lifespan” [6]. This site selection decision making process can be considered a complex task because it requires accounting on different factors (qualitative and quantitative) that influence the selection of a potential off-grid site over other alternatives [7]. Unfortunately, as authors Rahman et al. [8] argue, “a large number of off-grid rural electrification projects have failed because the focus was given on technical installation without paying sufficient attention to the long-term sustainability”. Thus, a multi-criteria analysis is useful to address this multi-faceted decision problem while accounting on multiple dimensions [8,9].

In the literature, we find a variety of multi-criteria decision making (MCDM) methods that support different rural electrification decision making processes [8,9,10,11,12,13,14,15,16,17,18]. In particular, these multi-criteria approach are mainly incorporated in two types of studies: (1) studies that apply a technology assessment to select the most appropriate technology for the energy provision in rural communities based on the attributes that each potential technology exhibits [11,12,13,14,15,16,19]; (2) studies that evaluate potential site alternatives to implement rural electrification projects based on different criteria [8,9,10,18]. In both scenarios, the sustainability assessments require accounting on a set of indicators that are related to the SDGs and cover multiple dimensions of sustainability [20]. Some authors refer to these dimensions as ‘criteria’, a term that finds correspondence in the “set of factors that may be used to assess which of a range of options offers the greatest contribution to achieving sustainability objectives” [21].

Traditionally, the concept of sustainability is associated with the triple bottom line (TBL) framework, composed of three key dimensions of sustainability: social, environmental, and economic or principles [20,22]. However, we, along with some other authors [8,9,10,11,12,13,14,15,16,17,18], believe that the TBL is not sufficient to capture all the perspectives that affect the sustainability of rural electrification projects. Fuso Nerini et al. [10] argue that rural electrification in the Brazilian Amazon is associated with 16 criteria grouped under five dimensions, namely technical, economic, environmental, social and institutional/political. They applied a multi-criteria approach based on a simple weighting methodology (i.e., the direct method of Anchored Rating Scale (ARS)), where decision-makers were asked to quantify their preference through a participatory process. As a result, biomass systems and solar PV systems registered the highest scores in the final index, 77%, and 76%, respectively. Ghasempour et al. [17] present a review on a variety of multi-criteria decision-making techniques applied to select solar plant sites and technologies. This includes the application of Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) methods, Analytic network process (ANP), Analytic Hierarchy Process (AHP), the Preference Ranking Organization Method for Enrichment of Evaluation (PROMETHEE) method, among others. Rahman et al. [8] applied a Stochastic Multi-criteria Acceptability Analysis (SMAA) method and Monte Carlo simulation to aid the process of selecting the best option for expanding electricity access in a rural area of Bangladesh. In this case study, the author considered 24 criteria values grouped into five sustainability dimensions. As a result of this process, the solar PV and biogas plants were found as the most preferable alternatives with 59% and 41% acceptability in their first rank, respectively.

Pérez and Díaz [16] used the case study of Colombia to develop a methodology to assess the implementation of solar PV projects in rural areas of Colombia. They assessed all the country’s departments using an AHP and considering techno-economic, social and environmental risk dimensions. The evaluation was based on data from the National Survey on Living Conditions in Colombia (NSLCC) and the Institute of Hydrology, Meteorology and Environmental Studies (IDEAM). As a result, a list of eight departments represented the prioritised regions to implement solar power projects in the country. Cherni et al. [9] designed and implemented a multi-criteria sustainable rural energy decision support system (SUREDSS) to select the most appropriate set of energy options for providing sufficient power to a rural and remote community in the department of Arauca, Colombia. Based on this multi-criteria approach, they ranked the possible energy system options (i.e., Micro-hydro, solar PV, Biogas, Hybrid diesel-biogas, Hybrid diesel-solar and Hybrid diesel-hydro) through four dimensions: physical, financial, natural, physical and human assets. As a result, the SURE program identified the best technology option based on a total score for each possible energy alternative where the highest given rating is 1 and the lowest is 0.

Overall, there is not a unique reference for the type of MCDM methods, dimensions and criteria used to implement sustainability assessments within the rural electrification sector. In this paper, we use the MACBETH approach and software to assess the suitability of different potential off-grid sites for implementing off-grid solar PV projects, based on a set of sustainable dimensions and key criteria. MACBETH is a non-numerical approach designed to aid decision-makers to quantify the relative attractiveness of options [23]. It has been successfully applied in several real-world case studies including the area of energy technology choice [24,25,26,27,28,29]. However, this paper constitutes the first study applying the MACBETH approach to the off-grid site selection decision problem.

Unlike other existing methods that assist rural electrification decision-making, the MACBETH approach uses a simple dialogue process in which the decision-maker (DM) carries out pairwise comparisons and gives only qualitative judgments about the differences in preferences of certain options [20,30,31] As a result, this procedure may be suitable for a DM who does not like to quantify preferences or to express his preference judgments numerically. Also, a particularly useful aspect is that MACBETH allows graphical representations of the performance profiles of the assessed options, making it possible to visually compare the attributes of each alternative in each criterion.

One of the limitations of this approach appears when evaluating more than five options. When the number of options (or the level of descriptors) is high, the MACBETH method might become an exhausting process to the DM, who must provide judgments of the difference of attractiveness between all the possible options and thus, resulting in a time demanding process. It is also worth mentioning that some of the methodological steps followed in this paper, are not exclusive to the MACBETH methodology, such as the procedures for developing and validating constructed descriptors of performance. Instead, they can be combined with other value function assessment approaches. Doing so, however, may sacrifice the added value of using a qualitative pairwise comparison approach.

In this paper, we illustrate the application of this multi-criteria model by evaluating the performance of three potential off-grid sites (i.e., Bolivia, Peru and Colombia) to implement off-grid solar PV projects. We focus on addressing the following questions: Which factors determine the long-term sustainability of off-grid electrification projects in rural and remote communities? and Which region represents the most sustainable option for implementing off-grid solar PV projects? To provide some answers, we calibrate the model, following a participatory process with the input of a DM with extensive experience in the rural electrification sector. As a result, we identify a set of sustainable dimensions and key criteria that influence the decision-making of selecting the most appropriate locations to implement off-grid solar PV projects. In addition, we reflect on the concept of sustainability assessment for site selection by estimating a composite score that groups each alternative into three categories: Highly sustainable (score > 100), Sustainable (0 < score < 100) and Unsustainable (score < 0), considering that each place offers a different level of attractiveness of investment for the DM.

The results from this study offer guidance to investors seeking to identify the most suitable sites for implementing off-grid solar PV projects in rural and remote areas, while accounting for sustainable criteria. Also, these results can help policymakers to identify and prioritise geographical areas for the provision of electricity access in developing countries.

2. Materials and Methods

2.1. Multi-Criteria Modelling

In the present study, we applied the MACBETH approach, to evaluate the performance of three potential locations for implementing off-grid solar PV projects in terms of a set of key criteria defined by the DM.

Figure 1 illustrates the main phases of the multi-criteria modelling implemented in this study.

2.1.1. Problem and Model Structuring

As mentioned above, private firms face the decision problem of valuating options for expanding their operations to other countries. In a setting where uncertainty is the norm, decision-makers seek information that enables them to select a foreign market from a set of possible alternatives, based on different aspects that guarantee the sustainable operation of a particular project, providing provide a favourable investment environment for the investor. Relevant criteria concerning this choice include, for instance, the level of poverty in the region, the political and economic environment of the countries, among other aspects. Therefore, a MCDM is technically adequate for dealing with this problem [23,24,25].

To illustrate the real-world application of the MCDM model, we used the MACBETH approach and software to evaluate the performance of three potential locations for implementing off-grid solar PV projects in developing countries. Following a participatory process, the model was calibrated with the input of a decision-maker (DM) with extensive experience in off-grid energy projects (a co-founder and former CEO of Kingo (Kingo is a provider of prepaid solar energy that has installed over 50,000 solar home systems (SHS) in off-grid communities of Guatemala and Colombia, between 2013 and 2020)) [32]. Thus, these judgements represent the point of view of a private company that makes investments in the rural electrification sector of developing countries. The point of view of other stakeholders, including academia, governments, or NGOs, are not considered in this study.

After defining the decision problem and its context, the following step was to define the screening criteria (i.e., any process that reduces a large set of alternatives to a smaller set that (most likely) contains the best choice [33]). During the meeting with the DM, we performed a preliminary screening analysis to select or reject possible options. In this case, based on the company’s commercial experience and the 2018 electricity access rates (acceptance criterion), we selected the three South American countries with the lowest rates of rural electrification, namely Bolivia, Peru and Colombia, as suitable alternatives to implement off-grid solar PV projects (see

Table 1. Comparison of electricity access rates and other development indicators associated with the selected countries (as of 2018). Source: Adapted from [36,38,39,40,41,42,43].

	Countries under Assessment
	Bolivia	Peru	Colombia
Population (Million)	11.35	31.99	49.65
GDP per capita ($ US)	3548.59	6941.24	6667.79
Multidimensional Index Poverty (MPI) a	Value = 0.094	Value = 0.029	Value = 0.02
	H = 20.4%	H = 7.4%	H = 4.8%
	A = 46%	A = 39.6%	A = 40.6%
Rural population (%)	30.57	22.09	19.22
Access to electricity (%)
Total (%)	92.8	92.9	96.5
Urban areas (%)	99.5	96.8	99.5
Rural areas b (%)	78.3	79.6	86.5
People without electricity access (Million)	0.9	1.2	1.7
Solar energy installed capacity (MW)	70	324	86
Universal electricity access target (year)	2025	2030	2030

^a MPI = H∙A where H is the proportion of multidimensionality poor people in the population and A is the intensity of poverty and reflects the average proportion of the weighted component indicators in which multidimensionality poor people are deprived. Estimation based on survey data: Colombia (2015/2016), Peru (2018), and Bolivia (2008). ^b Proportion of rural population with access to electricity.

). These locations also represent interesting business opportunities for off-grid companies since there has been a strong commitment from the local governments to implement national policies, regulations and programs aiming at increasing the electricity provision in rural and remote communities mainly through stand-alone PV systems and the grid extension [5,34,35]. The greatest cases correspond to Bolivia and Peru, where rural electrification rates increased in about 65% between 2000 and 2018 [36]. In the Colombian context, as of 2020, there were about 13,000 stand-alone PV systems installed in off-grid areas of the country. In addition, it is estimated that about 4821 rural and remote households will receive the electricity service through this type of energy solutions. by 2021 [37]. Table 1 presents a comparative review of the key development indicators between the three countries.

Within each country, three states were selected for the sustainability assessment (see Figure 2): La Guajira (Colombia), Loreto (Peru) and Pando (Bolivia). These regions have the lowest electricity access rate in each selected country: 58.81%, 76.6% and 83.1%, respectively [38,39,40].

As the technology energy solution to be implemented in each region, we considered the installation of standalone PV systems at the household level. Rural electrification programs within the selected countries are promoting the installation of this type of energy solutions, since extending the grid to rural and remote areas is not an economically viable option [44,45,46]. In addition, according to the International Energy Agency (IEA), “nearly 60% of additional generation needed to achieve universal access to electricity by 2030 will come from off-grid installations, which can be either mini-grids or stand-alone systems” [47,48]. In this case, we consider a stand-alone system with a set of two (2) solar panels of 370 Wp each, a charge controller, a battery pack, an inverter, an electrical panel board, and the associated electrical installation, as illustrated in Figure 3.

After an interactive and learning process, the model structuring began with the analyses of the DM’s fundamental points of view (FPV). These are the relevant attributes used to evaluate each off-grid location alternative according to the DM’s values. This approach is called “value-focused thinking” [49].

These factors were identified by the DM based on his experience and they are related to the main issues to overcome and the type of risks the investors need to mitigate in order to guarantee the long-term sustainability of rural electrification projects [50]. As a result, 13 evaluation criteria were grouped into five dimensions of sustainability: economic, commercial, technical, environmental and social. Figure 4 provides an organised overview of these factors using a logical framework known as ‘value-tree’.

It is important to mention that despite being a highly interactive process, we only present here the final outcome—the ‘requisite’ model [51]. This means that several adaptations were necessary to guarantee that the form and content of the model were sufficient to provide satisfactory and undisputed answers to the DM’s questions.

After pinpointing the fundamental criteria, the next step consisted of associating a measure (qualitative or quantitative) of the extent to which each criterion can be satisfied, namely a descriptor of performance that operationalises each criterion. According to Bana et al. [52], a descriptor of performance is an “ordered set of plausible impact levels or performance levels associated with an evaluation criterion”. Considering the criteria underpinning this decision, we opted to define quantitative descriptors to operationalise a criterion, where possible.

Table 2. Selected evaluation criteria and associated descriptors of performance.

Dimension	Criteria	Descriptors of Performance (Brief Description)	Unit of the Descriptor	Relevance
(E) Economic	(E1) 2018—GDP per capita	Estimated by the country’s gross domestic product	US $	The extent to which a higher GDP provides more confidence about the economic performance of the country. This indicator allows for making cross-country comparisons of average living standards and economic wellbeing.
	(E2) 2013–2018 Average GDP growth	Estimated by the average of the GPD during the last five years	%	The extent to which the potential project is affected by the economic stability of the country. It reflects the country’s efforts to promote economic growth.
	(E3) 2018—Poverty Index (PL)	Estimated by the percentage of people living in extreme poverty (currently set at $1.90 a day)	%	The extent to which the poverty condition of the families undermines the collection of the payments and increases the customer churn rate.
	(E4) 2018—Financial Inclusion	Estimated by the percentage of people having access to at least one financial service (including bank account, credits, loan, equity, and insurance products)	%	The extent to which the potential project is affected by the lack of evidence of the consumer’s credit history. It represents a risk to the energy provider who assumes the consumer’s responsibility in repaying debts.
(C) Commercial	(C1) Potential off-grid households (without electricity access)	Estimated by the total number of potential households without access to electricity	Number of households	The extent to which the project is impacted by the new potential installations within a community. The higher the number of households, the better.
	(C2) Dispersion index (potential off-grid households/area)	Estimated by the population density of the potential site. In this case, the potential number of off-grid households per unit area	Number of households/km2	The extent to which the project sustainability is affected by low-density communities, namely how scattered is the community within each potential region. This index increases/decreases the total installation and operations and maintenance (O&M) costs of the project
	(C3) 2018—Rural Mobile phone ownership	Estimated by the percentage of people (>5 years old) living in rural areas, that own a cell phone	%	The extent to which the project’s commercial operation is affected by the lack of local people owning a cell phone, required for the monetary transactions. It is important to guarantee the success of the energy prepaid model used by most off-grid energy companies.
	(C4) Solar PV Investment-National rural electrification plan	Estimated by the total investment for rural electrification projects based on solar PV energy and included in the National rural electrification plan	US $M	The extent to which the local government is investing in solar energy for rural electrification and in particular, if there are available funds to provide financing for solar PV projects in the country.
(T) Technical	(T1) Solar energy potential	Estimated by the Global Horizontal Irradiance (kWh/m2). It is the total amount of shortwave terrestrial irradiance received by a surface horizontal to the ground	kWh/m2	The extent to which the project is impacted by the availability of local solar energy resources. In rural and remote areas, having robust systems with high reliability is an important advantage due to the logistical difficulties in providing maintenance.
(Env) Environmental	(Env1) Protected Areas	Estimated by the percentage of conservation areas that receive protection because of their recognized natural, ecological, or cultural values	%	The extent to which the project is impacted by the percentage of protected areas (e.g., natural parks, indigenous settlements or marine reserves), increasing/decreasing the installation and O&M costs of the project.
	(Env2) Average terrain elevation	Estimated by the altitude (above sea level) of the project potential location	m.a.s.l.	The extent to which the commercial operation of the project is affected by the topography of the project location, measured by the easiness of accessing the community due to its terrain elevation.
(S) Social	(S1) Rural Illiteracy rate	Estimated by the percentage of the population (>15 years old) that can read and write	%	The extent to which the commercial operation of the project is affected by the lack of local technicians with a minimum level of education. Hiring local technicians not only reduces the installation and O&M costs but also increases community acceptance.
	(S2) Security Level	Estimated by the Global Peace Index which measures the level of peacefulness within a country	points	The extent to which the potential location provides a good level of safety for the technical staff installing/operating the systems in the community. Some indicators included in this score are the number and duration of internal conflicts, the number of homicides per 100,000 people, among others.

presents a brief description of each descriptor, its unit and its relevance to the sustainability assessment of potential off-grid locations. This latter aspect reflects on why it is important to the DM to consider each FPV and to what extent. The attractiveness of each potential location alternative, Bolivia, Colombia and Peru, is appraised by the DM within the corresponding performance scale. For instance, a higher priority is given to an area with a higher GDP because it gives the DM more confidence that the projects will be successful. A similar rationale is behind the concept of the other FPV: the alternatives having a lower poverty index, a higher financial inclusion, a greater number of households without electricity access, or a lower dispersion index are preferred.

However, in this model, the potential off-grid locations presented different performances (the performance profile of each option is presented in Table S2 of the Supplementary Material of this paper) within the different evaluation criteria, thus, the process of choosing the option with the best performance was complex. If there had been an option that outperformed the others in every criterion, the choice would have been intuitive and there would have been no need to apply the M-MACBETH software to support the decision.

2.1.2. Building the Additive Evaluation Model

After the classification of the options according to each FPV, we proceeded to the construction of the multi-criteria additive evaluation model. Following the MACBETH approach, we built the value functions for each FPV. This process was done by requesting the DM to define two reference levels (or anchors) in terms of performance for each FPV: a reference of “Neutral” performance and a reference of “Good” performance, explaining that the former refers to the level below which there are the unattractive options or from which the DM set a minimally satisfying performance and the latter refers to the level above which there are the most attractive or outstanding options with satisfying performance. The partial values ($v_j$) of these anchors for a criterion $j$ are defined as: $v_j\left(good\right)=100$ and $v_j\left(neutral\right)=0$. These references helped us to investigate whether each option had an undesirable (worse than Neutral), satisfactory (from Neutral to Good) or an outstanding (better than Good) performance [53]. As a result,

Table 3. Reference levels (‘good’ and ‘neutral’) and attractiveness of each criterion.

Dimension	Criteria	Reference Values		Ranking of Importance
		Neutral (Acceptable)	Good
Economic	(E1) 2018—GDP per capita	5000	6000	4
	(E2) 2013–2018 Average GDP growth—5 years	3	7	3
	(E3) 2018 Poverty Index (PL)	30	20	5
	(E4) 2018 Financial Inclusion	60	80	8
Commercial	(C1) Potential off-grid households (without electricity access)	40,000	80,000	1
	(C2) Dispersion index (potential off-grid households/area)	0.5	1	2
	(C3) 2018 Rural Mobile phone ownership	50	60	6
	(C4) Solar PV Investment- National rural electrification plan	200 M	400 M	9
Technical	(T1) Solar potential	Medium	Very High	11
Environmental	(Env1) Protected Areas	10	0	12
	(Env2) Average terrain elevation	110	50	13
Social	(S1) Rural Illiteracy rate	10	5	7
	(S2) Security Level	4	0	10

presents the reference levels for each descriptor.

Afterwards, to build the value functions for each FPV, the DM was requested to give a qualitative judgment on the difference of attractiveness between levels of performance of the respective FPV. To model the differences in preference, the DM used one of the seven MACBETH categories for the judgments, namely: “extreme”, “very strong”, “strong”, “moderate”, “weak”, “very weak”, and “no” difference. If unsure about the difference of attractiveness, the DM could choose more than one qualitative rating. If any two options were considered to be equally attractive during the ranking process, they were assigned a “no” in the matrix [30]. Then, after checking for no remaining inconsistencies, these judgments were processed by the M-MACBETH and the program proposed a quantitative value function for each FPV (See Figure S1 of the Supplementary Material). This quantitative value function represents the judgments of the DM and assigns value scores to the levels of each performance descriptor relative to the fixed scores of 0 and 100 (i.e., the neutral and good reference levels in the model). As a simplification (and to expedite the decision meetings), this model assumes a linear relationship between performance and score, namely a linear value function that measures the attractiveness of each site alternative within each FPV [31]. This additive aggregation is represented by the value function model defined in Equation (1):

$$\mathrm{V}\left({\mathrm{x}}_{\mathrm{i}}\right)={{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{j}}{\mathrm{v}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}\right),$$

(1)

With,

$${\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{j}}=1$$

${\mathrm{w}}_{\mathrm{j}}>0$

${\mathrm{v}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}^{+}\right) ={\mathrm{v}}_{\mathrm{j}}\left(’\mathrm{good}{´}_{\mathrm{i}}\right) =100$

${\mathrm{v}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}^0\right) ={\mathrm{v}}_{\mathrm{j}}\left(´\mathrm{neutral}{´}_{\mathrm{i}}\right) =0$

$\mathrm{i}=1,\dots , \mathrm{m}$ and $\mathrm{j}=1,\dots , \mathrm{n}$

where $\mathrm{i}=1,\dots , \mathrm{m}$ represents each site alternative and $\mathrm{j}=1,\dots , \mathrm{n}$ designates each evaluation criterion (e.g., Dispersion Index); ${\mathrm{w}}_{\mathrm{j}}$ are the scaling constants or the weighting coefficient assigned to criterion $\mathrm{j}$ multiplied by the partial value score ${\mathrm{v}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}\right)$; ${\mathrm{x}}_{\mathrm{i}}^0$ and $\left({\mathrm{x}}_{\mathrm{i}}^{+}\right)$are, respectively, the neutral and good performance levels defined for each performance descriptor, and V$\left({\mathrm{x}}_{\mathrm{i}}\right)$ is the overall score of alternative${ \mathrm{x}}_{\mathrm{i}}$ that measures its global attractiveness. As a result, the most appropriate off-grid location results from max $\mathrm{V}$ in Equation (1).

2.1.3. Computing the Weighting Coefficients

In order to measure the added benefit of each potential off-grid location, we defined the weighting coefficients of each criterion that needed to be defined. According to Marques et al. [20], weighting coefficients are ‘scaling constants’ that transform partial scores (in each criterion) into global scores. For each criterion, these coefficients represent the importance that the DM gives to the swings in performance (e.g., from ‘neutral’ to ‘good’). Thus, after establishing how well each off-grid site alternative performs on each criterion, those performances must be combined or aggregated to have an overall score for each potential site. This stage of the model evaluation consisted of ranking the criteria by decreasing order of attractiveness, considering a switch from “neutral” to “good” reference level of each criterion. For this purpose, we used the MACBETH qualitative swing weighting process [54] and the following question was asked: “Presume that one of the alternatives is expected to have a low (‘Neutral’) performance on all 13 criteria; on which criterion would a swing from ‘Neutral’ to ‘Good’ be most attractive?” The same question was made for the second most important criterion, and so on until the last criterion was included. The final ranking is presented in Table 3. Then, the DM compared qualitatively the most important swing to each of the others and finally, followed by the comparison of each two swings consecutive in the ranking.

As each DM’s judgment was given, the M-MACBETH software automatically verified the matrix consistency and suggested judgments modifications that were used to fix the detected inconsistencies. Finally, the software proposed a set of weighting coefficients that were compatible with the DM’s qualitative judgments. With these weights and the partial value function ${\mathrm{v}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}\right)$ of each option for each criterion, the software also computed the global attractiveness of each option. This overall score resulted from the additive aggregation of the weights and performance of each option within each criterion, as defined in Equation (1).

2.1.4. Testing the “Requisiteness” of the Model

As presented in Figure 1, the last step of the multi-criteria modelling consists of validating the model. This process required performing a sensitivity analysis. Thus, we modelled some susceptibilities and assessed the model’s consistency. In particular, we evaluated how a change in any of the weights (within the allowed interval) would affect the overall scores of the options, and, to a greater extent, possibly change the model’s recommendation. To facilitate this process, we first asked the DM to examine and confirm the weights and then to identify the weights whose values he was not certain about. Finally, we validated and analysed the effect on the alternatives’ overall score.

3. Results and Discussion

In this study, we applied a multi-criteria decision analysis approach, namely an additive value model, to evaluate the performance of three potential site options for implementing an off-grid solar PV project. By using the MACBETH approach and software we could provide an overview of the performance profiles of the assessed options, rank each site alternatives according to a sustainability scale, identify the main factors influencing the site selection process and show the remaining challenges that investors and policymakers need to overcome to guarantee the sustainability of off-grid solar PV projects in developing countries. In this section, we present the main results of the implemented sustainability assessment process. These results only represent one type of stakeholder, namely the private sector or developers of off-grid solar PV projects.

Based on the weighting procedure presented in Section 2.1.3, the DM ranked the criteria, and with his qualitative judgments, the M-MACBETH software provided the numerical weights for each criterion, as illustrated in

Table 4. Weights of criteria and associated dimension.

Dimension	Weight (%)	Criteria	Weight of Criteria (%)
Economic	42.16	(E1) 2018—GDP per capita	12.71
		(E2) 2013–2018 Average GDP growth—5 years	12.71
		(E3) 2018 Poverty Index (PL)	11.52
		(E4) 2018 Financial Inclusion	5.22
Commercial	45.75	(C1) Potential off-grid households (without electricity access)	18.47
		(C2) Dispersion index (potential off-grid households/area)	16.52
		(C3) 2018 Rural Mobile phone ownership	8.91
		(C4) Solar PV Investment- National rural electrification plan	1.85
Technical	1.2	(T1) Solar energy potential	1.2
Environmental	1.53	(Env1) Protected Areas	0.87
		(Env2) Average terrain elevation	0.66
Social	9.36	(S1) Rural Illiteracy rate	7.83
		(S2) Security Level	1.53

.

The obtained weights have a direct influence on the final decision because they allow the DM to evaluate the most critical dimension and criterion affecting the selection of an off-grid market. The resulting matrix of judgments is presented in Table S4 of the Supplementary Material of this paper.

Accordingly, the sum of each criterion yielded to the total weight assigned to each dimension. Overall, the Economic and Commercial dimensions report the highest importance compared to the other dimensions: 45.75% and 42.16%, respectively. In contrast, the Technical dimension has the lowest effect on the final decision, weighing 1.20%. This result suggests that the Solar energy potential criterion was not considered a determinant factor to have a preferred option, mainly because the three site options exhibit a similar performance, having a very high potential (i.e., GHI > 4.5 kWh/m²) (The performance profile of each option is included in Table S2 of the Supplementary Material of this paper). Thus, none of the options outperforms the others in this criterion. Figure 5 shows the relative weights for each criterion within the five dimensions. The Potential off-grid households and the Dispersion index represent the criteria with the highest importance compared to the other key factors: 18.47% and 16.52%, respectively. This result evidence the importance of the project size (i.e., the number of households without electricity access) not only to guarantee financial viability but to prioritise the communities where the project could impact more lives. This value refers to the number of new connections that the company should make to meet its costs, that is, to break even and, according to the DM, it has a value between 5000 and 8000.

Regarding the Dispersion index, it was considered as the second most influencing criterion, because the installation of the systems and the associated installation and O&M costs depend on how scattered the potential community within each potential region is.

Figure 5 also evidences that the Protected areas (%) and the Average terrain elevation (m.a.s.l.) criteria have the lowest weights on the selection of an off-grid market: 0.87% and 0.66%, respectively. In this respect, it was clear that rural electrification investments must acknowledge the special geographic conditions of the regions where off-grid communities are usually settled. Therefore, an additional margin on the operation and maintenance costs is often included to cover the expenses of projects exhibiting logistical constraints. As a result, the obtained weights for these criteria suggest that no matter the off-grid project location is, investors must cope with the local environmental constraints, and thus, for this particular case, these two criteria have not a determinant influence (both weights sum up 1.53%) on the final selection of the potential site.

Having the relative weights for each criterion, it was possible to determine the associated sustainability assessment score for each option (i.e., La Guajira (Colombia), Loreto (Peru) and Pando (Bolivia)), across all the dimensions and criteria. In particular, Figure 6 presents the scores associated with each dimension by country. These scores represent the degree to which each dimension contributes to each option’s sustainability.

As presented in Figure 6, Colombia has the best performance in terms of the Commercial, Environmental and Social dimensions, which have a direct influence on the site selection in about 57% (See Figure 5), whereas Peru has a better performance in the Economic dimension. Figure 6 also shows that the Commercial, Economic and Environmental dimensions are determining factors between the worst options, making Bolivia less attractive than Peru.

Additionally, in order to provide a more structured approach to evaluate the prospective potential market, we addressed another question: “what are the biggest differences between the options, in terms of the criteria?” As a result, all the criteria were evaluated individually for each of the potential regions. reports the final values of the aggregate scores associated with each criterion, based on a 0 (neutral) and 100 (good) scale of reference.

Figure 7 displays more clearly the biggest differences between the positive and negative scores across the options. In particular, it reveals why Colombia has a better overall score. Also, it shows that Colombia and Peru are mainly positively scored options and most of the criteria are weighted above neutral, especially in Colombia. By paralleling the scores of Colombia and Peru (the two best options), we observed that Colombia has a better performance in terms of the Potential off-grid households and the Dispersion Index, a similar performance for GDP per capita, Solar potential, and Security level. The only downside for Colombia is the Average GDP growth that is below the neutral, representing a low increase in economic performance compared to Peru and Bolivia.

The Rural Illiteracy rate criterion is a common punishing factor between options, which shows the low potential of finding skilled personnel in the three regions. Also, the Dispersion Index has another negative effect on the global score. These two FPV have an impact weight of 7.83% (Rural Illiteracy rate) and 16.52% (Dispersion) and they will be evaluated in the sensitivity analysis.

On the other hand, by comparing Bolivia and Peru, we noticed that Bolivia only has six positive weighted criteria (i.e., values in Average Terrain Elevation (Env2), Solar Potential (T1), Solar PV investment (C4), Mobile phone ownership (C3), Potential off-grid households (C1) and Average GDP growth (E2)). In particular, the GDP per capita criterion with an overall weight of 12.71% (see Figure 5) has the worst effect on Bolivia’s global score, representing an entry barrier and a less appealing option for the DM.

As previously illustrated in the value tree shown in Figure 4, there are five primary dimensions for the off-grid site selection problem. Looking at the results, it was clear that Colombia was the best option in terms of the Commercial, Environmental, and Social dimensions. However, there were some divergences on the Economic dimension with Peru having a better performance in this respect. Figure 8 presents an XY map to compare the options’ scores in a bi-dimensional graph between two criteria or dimensions. In this case, the line that connects the options maximizes the score for the Economic dimension. The previous analysis showed that although there were three options, only two of them had positive overall results, so it was expected that the efficient frontier only took Colombia and Peru into account. As mentioned earlier, the weight of the Economic dimension is 42.16% (12.71% from GDP per capita, 12.71% from Average GDP growth, 11.52% from Poverty Index, and 5.22% from Financial Inclusion). We used this result as a starting point for sensitivity analysis of the best two options found.

On the other hand, the application of the model enabled the DM to appraise the potential alternatives under consideration. After considering the scores for each criterion and the associated weights, the model proposed Colombia as the most attractive option given the DM’s judgments. This country has the best score (98.23 on a scale of 0 to 100), leading over Peru (the second-best option) by about 27 points.

Based on the sustainability score attained by each country and also on the definition used to set the good and neutral levels of reference associated with each criterion, we categorised the countries’ sustainability as follows [20]:

• Highly sustainable: if the potential site has a score above 100 (i.e., more sustainable than a country that has a performance equal to the ‘good’ level in all criteria).
• Sustainable: if the potential site has a score between 0 and 100 (i.e., it represents the minimum acceptable level)
• Unsustainable: if the potential site has a score below 0 (i.e., less sustainable than a country that has a performance equal to the ‘neutral’ level in all criteria).

As we present in

Table 5. Global Scores for the most appropriate market option and the associated level of sustainability.

Market Option	Global Score of Appropriateness *	Country’ Sustainability
Colombia	98.23	Sustainable
Peru	71.56	Sustainable
Bolivia	−28.46	Unsustainable

* The larger the score, the most appropriate the site, based on a 0 (neutral) and 100 (good) scale of reference.

, the off-grid solar PV project in La Guajira, Colombia, has the best performance in the sustainability assessment, namely, it represents the highest sustainable option for a project investor, among the remaining site options. Table 5 also shows that not all the alternatives have a positive evaluation score, which suggests that Bolivia is a low viability alternative or an unsustainable option for the DM, with a score of −28.56. None of the options has a highly sustainable score, meaning that none of the countries represents an outstanding option, however, Colombia and Peru, are the alternatives with a minimally satisfying performance according to the DM’s judgments of preference.

3.1. Sensitivity Analysis

We also performed a sensitivity analysis to validate the consistency of these outcomes. The purpose of sensitivity analysis is to refine the decision model, with the ultimate objective of obtaining a requisite model, which when subjected to sensitivity analysis, yields no new intuitions [51]. There are several types and tools of sensitivity analysis. Simple one-way sensitivity graphs can be carried out using tornado diagrams and spider graphs. Also, sensitivity analysis simultaneously accounting for many variables can be done using Monte Carlo simulation models. In this study, we used the M-MACBETH software to assess the consistency of the final ranking of the countries’ alternatives.

We tested the model using two scenarios where we studied the susceptibility of the overall score of the options, by making small variations of one criterion weight at a time. This analysis was performed on the positive overall scored options (Colombia and Peru). Figure 9 shows the modifications that occur from modifying the weight of a criterion while maintaining the proportion between the remaining criteria weights. Figure 9a shows that the model’s result is not sensitive to variations on the weight of the Potential off-grid households. This means that the relative global scores of the countries’ alternatives remain unmodified, and Colombia continues to be the most sustainable option. This is relevant when considering that this criterion has the highest influence in the evaluation model, i.e., the highest weight (18.47%).

As previously perceived, Colombia is always preferable to Peru in terms of the Commercial, Environmental, and Social dimensions, but in terms of the Economic dimension, there were some doubts about the sensitivity of the weight obtained. In this dimension, Colombia had similar behaviour in the following criteria: GDP per capita, Average GDP growth and Financial Inclusion, compared to Peru. Figure 9b shows the Economic dimension susceptibility in terms of the Poverty Index. If we increase the weight of this criterion from 11.52% to 37.8%, the final overall relative score changes and the model proposes Peru as the most attractive alternative. However, this is less likely to occur since there are other economic important criteria to consider in the decision-making process, such as the GDP per capita, and the Average GDP growth representing a total weight of 25.42% on the final selection.

Both situations were validated with the DM and no change in weights was required. It is worthy to mention that in terms of the Commercial, Environmental and Social dimensions, both countries had divergent scores for the set of criteria, so no matter the weight, Colombia is always the most suitable site option.

There are two possible areas for further work. First, we propose to validate the obtained set of criteria, by calibrating this model with the input of other decision-makers (e.g., other off-grid energy utilities, actors from the local government and experts from the academic sector) to reflect a diversity of stakeholder views. Second, we suggest to include additional factors that were not considered in this study, namely the following criteria: Community acceptance (i.e., to guarantee the successful implementation of an off-grid system, the project should fit well into the socio-cultural context of the community) and Political environment (i.e., the extent to which the potential business is impacted by the political stability of the country), which could also contribute to the long-term sustainability of off-grid projects. Finally, it would be useful to provide a detailed discussion of how to make those regions not selected, more attractive to investors.

3.2. Policy Implications

The definition of sustainable rural electrification programs and policy guidelines depends on considering the point of view of different stakeholders, such as utility companies, providers, academia, and the public sector. Having each sector’s insights creates a collaborative visualisation and understanding of the regulatory, financial, economic, environmental, social, and technical needs to guarantee the long-term sustainability of off-grid projects, regarding of the type of technology.

In this study, we focused on identifying key sustainability criteria for implementing off-grid projects based on the private sector perspective. These indicators are of different relevance depending on the key actor’s perspective. For instance, for private companies, a higher GDP per capita gives more confidence that the off-grid project will be successful. In contrast, the public sector could give more relevance to a lower GDP per capita, because the government wants to have a maximal impact on poverty.

In consequence, it is essential to incorporate the private sector’s concerns into the rural electrification policy and, in particular to the process of designing rural electrification mechanisms that incentivise the private capital participation in off-grid projects by offering an adequate investment environment for private companies. The MCDM method carried out in this paper can be replicated to assess other potential off-grid locations and thus, to roll out electricity access programs in developing countries.

4. Conclusions

Selecting the most appropriate site for implementing off-grid solar PV projects, is one of the key decisions that private investors face within the rural electrification sector. In this paper, we implemented a multicriteria model to support a decision-maker representing the private sector’s point of view, in evaluating the performance of three off-grid site alternatives against 13 criteria grouped in 5 sustainability dimensions.

Through analysing the three options with the MACBETH approach and software, we found that Colombia and Peru are the most sustainable alternatives based on their associated global score (98.23% and 71.56%, respectively). In addition, the number of Potential off-grid households is the criterion with the highest influence on this evaluation model, with a weight of 18.47%. Overall, the Economic and Commercial dimensions have a direct influence (in about 88%) on the decision making of selecting the most appropriate off-grid location to implement stand-alone solar PV projects.

In summary, the main contribution of this paper is threefold: first, we provided a useful roadmap for MCDM modelling through participatory processes, with a practical example on how to structure a model considering the case of developing countries, and in particular, the context of off-grid rural electrification projects. Second, we identified key criteria and associated descriptors of performances that contribute to achieving sustainability in the off-grid energy sector. We integrated two additional sustainability dimensions to the TBL approach, namely the technical and commercial criteria. These additional dimensions build upon the technical and non-technical features that influence the decision-making of selecting the most appropriate off-grid site to implement solar PV projects, based on the private sector’s point of view. Third, our general findings may offer some direction for project developers and potential investors willing to undertake their own sustainability assessments. In particular, this methodology can be applied to evaluate other regions, namely, to identify and prioritise geographical areas for the provision of electricity access.