New Methodological Approach for Performance Assessment in the Bioenergy Field

Abstract

Bioenergy, along with other renewables, has always played its part in the world’s energy transition. Tracking the progress to meet specific goals has long been tackled and led to performance evaluation in the field. The present study aims to contribute to this area with a performance assessment framework in the bioenergy field. It comprises 16 European countries and 30 indicators assigned to three dimensions: innovation, efficiency, and sustainability and it follows a well-established methodology. For enabling country-to-country comparison, five maps are designed for better illustration. The country performance ranking is one of the main outputs of the analysis, revealing the outperformers and the weakest countries from its bottom half, as well as the particularities of countries scoring on each of the three dimensions. The policy recommendations and study limitations represent the most relevant part of the conclusions.

1. Introduction

Ever since society has called for a more sustainable life, renewable energy has been hailed as a reliable alternative to traditional energy and as a key to designing paths for achieving the goals of the Paris Agreement, for creating a circular economy and for playing its part in the energy revolution. The shift to renewables triggers a repositioning in terms of infrastructure (industrial and residential), technology, production processes, policies, and strategies.

At the macroeconomic level, the idea of performance assessment refers to verifying the extent to which a country carries out its activities to meet targets set at the national, regional or global level. In the field of energy, the performance study involves finding dimensions that contribute to optimizing key elements, such as energy consumption from renewable sources, energy efficiency, less polluting transport, energy security, and infrastructure needed to support the transition to a new energy era.

Developing a performance assessment framework in the bioenergy field is a challenge, a process which, once completed, can provide a tool with multiple uses. Such a tool is capable of revealing a transparent view of the bioenergy field performance and of facilitating comparisons among countries. At the same time, it may offer relevant information on the current status of development in this field and keep track of changes in the performance level in time.

The present paper is structured as follows: first, a conclusive review of the relevant literature is conducted, revealing contributions in the bioenergy field from an international perspective. Second, a well-developed methodological section explains the modelling assumptions for creating the performance assessment framework. The contributions made by this paper are discussed in the results section, while both policy implications and further research directions are provided in the conclusion.

2. Literature Review

Studies on performance evaluation at national level can be revealed for health systems where the concept is understood and treated in terms of five dimensions: quality, efficiency, access, sustainability, and equity [1], or in terms of only three components: effectiveness, equity, and efficiency [2]. Moreover, other research efforts provide indexes designed for influencing stakeholders’ decisions in a specific field. For instance, the Environmental Performance Index (EPI) is a biannually issued index of researchers from Yale University, which “ranks countries’ performance on high-priority environmental issues in two areas: protection of human health and protection of ecosystems” [3]. Also, as an example, the Aggregated Energy Security Performance Indicator (AESPI) is used to describe Thailand’s energy security status. It is calculated using 25 indicators on three dimensions (economic, social, and environmental), and among them, only two indicators are characteristic to renewable energy [4]. The World Energy Council has developed an indicator called the Energy Trilemma Index to study performance at the national level, taking into account three dimensions [5]: energy security, energy sustainability, and energy equity, each with a 30% weighting in the overall score, while the remaining 10% is added from three other elements: macroeconomic stability, a favorable environment for investment and innovation, and last but not least, governance.

In order to describe the methodology of our paper, it is important to highlight papers that describe the high impact of bioenergy in the world. Bioenergy is a way to reduce the amount of carbon dioxide emitted as a result of economic activities carried out in each country and is closely linked to innovation in technology. Based on the process of technological development, bioenergy has come to play an important role in increasing energy security, developing rural communities, and increasing household incomes [6], which ultimately culminates in reducing emissions from electricity production [7]. At the base of bioenergy are various types of energy that come from the conversion of natural, biological sources available on a renewable basis [8]. From this point of view, bioenergy brings into question traditions specific to the agricultural sector and offers solutions on waste management, elements with an impact on the efficiency of bioenergy production and use [9].

Besides natural, biological sources, bioenergy is also obtained from the conversion of waste to valuable energetic products, such as refuse-derived fuel or RDF obtained through the pyrolysis and gasification of municipal solid waste [10]. Bioenergy production has evolved from wood burning to hydrogen production through modern conversion processes, such as thermo-catalytic conversion of glycerol (a by-product resulted from biodiesel production) [11] or CO catalytic conversion of tar rich synthesis gas (resulted in a biomass gasification plant) [12].

At the EU level, attention has been focused on bioenergy for a decade now. The Directive 2009/28/EC put forth the objectives regarding the direction of the European Union towards environmental protection, establishing the achievement of renewable energy for each member of a share of 20% of the total energy consumption until 2020 and a share of 10% that biofuels should have in the consumption of gasoline and diesel in transports until 2020 [13]. Given the environmental problems that have felt their effects from one year to another, in 2016, the European Commission updated directives from 2009, establishing the achievement of a share of renewable energy of over 27% of total energy consumption by 2030, along with new criteria on sustainability [14].

Moreover, the environmental challenges we face are not the only elements that put pressure on our way of life. The growth of the planet’s population, the growing demand for raw materials, natural resources, and food are also factors with a strong impact on all mankind. Karmenu et al. [15] emphasize that renewable resources are consumed twice as fast as the capacity of ecosystems to recover. In order to meet all the challenges, the existing linear production model at the level of EU member states must be replaced with a new, circular model, to deal with the three dimensions of sustainable development: environmental protection, sustainable economy, and efficiency. The convergence element of these dimensions is renewable energy and the recycling of all materials, elements closely related to bioenergy [16]. In this transition from the linear economy to the circular economy, the bioenergy sector will develop accordingly; this will be produced from the combination of the residual biomass with the waste resulting from human activities [7,17]. In other words, the circular economy will allow, at the level of the European Union, the provision of resources for the production of bioenergy, an element with added value whose impact makes all these measures respond to the environmental challenges that the world is currently facing [18,19].

Taking into account all these elements, as many authors explain [14,20,21], European countries have begun to transcribe directives through their own programs, supplementing bioenergy production and moving towards renewable energy to meet both environmental and energy demand challenges. In this case, as the importance of bioenergy is growing, at the level of the European Union, in 2016, bioenergy represented over 59% of renewable energy sources used for final gross energy consumption [22]. In order to supply electricity to the entire population, a significant amount of greenhouse gases are produced, the effect of which is significant in increasing the concentration of pollutants in the atmosphere [23]. In this sense, more and more countries and international bodies are trying to use technologies and alternative energy sources that lead to reducing emissions, and thus to reducing pollution, factors that along with revitalizing rural areas and improving energy security make energy systems bioenergy to be in the public eye for decades to come. Bioenergy has three characteristics that favor the decision regarding its use [24]:

• High flexibility—arranging the space for bioenergy does not raise significant logistical problems. Biomass and biofuels have the characteristic to be more easily stored than other known forms of renewable energy, achieving a high level of flexibility. Moreover, in order to produce biofuels, we can use any organic origin material, so it is not necessary to bring biomass from a great distance or remote areas [25].
• Potential for integration in existing energy systems—the use of bioenergy will not lead to the destruction of existing infrastructure; instead, the infrastructure will be preserved being modified only the primary source of energy production.
• Attractiveness—bioenergy is seen in the literature as an alternative desired by all countries in the world, regardless of the level of economic development. Silveira [26] and Roberts [27] stated that bioenergy can create opportunities for regional development while providing energy in a sustainable way, being a solution for economic, environmental and political security. In addition to this, according to REN 21 [28], there are 179 countries that have adopted policy measures in order to remove barriers related to the development of renewable energy.

In the last 10 years, in order to measure and evaluate the environmental performance of different types of renewable energies, the Life Cycle Assessment (LCA) method has been used [29,30,31,32,33]. Some research has confirmed that the benefits associated with bioenergy such as reducing greenhouse gas emissions or ensuring energy needs have materialized, and these things have been captured by researchers over several periods [34,35,36]. In recent years, the LCA method has been used to determine and evaluate on the one hand how climate change is mitigated, and on the other hand what is the volume of fossil fuel savings when wood-based electricity replaces electricity produced with the help of fossil fuels [32,37].

This paper extends the analysis of bioenergy efficiency, achieving a multidimensional analysis of bioenergy performance seen through three dimensions: innovation, efficiency, and sustainability.

3. Methodology

The performance assessment in a specific field is a process described by a particular feature: multidimensionality related to the components included in the analysis framework. In general, these components reflect various economic, social, and environmental aspects, have diverse measurement units and are interpreted using different criteria and scales. As such, assessing the performance by using a set of data as described above may be difficult if not following some generally valid rules. Many studies follow the steps developed by The Organisation for Economic Co-operation and Development (OECD) in 2008.

The present research methodology includes the following steps, in accordance with the newest approaches in the renewable energy field and index construction methodology [38,39,40,41], as shown in Figure 1.

To our knowledge, there are few indexes designed for performance in the renewable energy field, such as the Environmental Performance Index (EPI) [3], the Aggregated Energy Security Performance Indicator (AESPI) [4], Renewable Energy Responsible Investment Index (RERII) [40], and Renewable Energy Country Attractiveness Index (RECAI) [42], but none directly related to bioenergy performance. We were able to find the Nexus index [43], which deals with the efficiency of biofuels production from a specific crop and Renewable Energy Sustainability Index [44] which reveals the sustainability level achieved by countries in the renewable energy field. However, the performance concept, as we defined it in the previous section of the paper, is more complex, including efficiency, and sustainability as components. Thus, the novelty of this research lies in the construction of the performance assessment framework through seven phases, as presented in Figure 1, and in obtaining, in the end, a new analytical tool: The Performance Index in the bioenergy field.

3.1. Structure of the Performance Assessment Framework

Given the description of the performance concept detailed in the previous section, the first step was to decide on the dimensions that will design the performance index.

The first chosen dimension is innovation, which has a major role in bioenergy field development, as shown in recent research of the literature [45,46]. Innovation is essential for at least two main reasons: (1) it is vital for the process of obtaining bioenergy, as the conversion of biomass into biofuels requires high-performance and cost-effective technologies; (2) it has great implications in the distribution and supply of electricity and heat from biomass processing.

The second dimension to be considered is efficiency, given its undoubted role in the economic and social environment in guiding projects contributing to the fulfillment of sustainable development objectives and to reaching targets in specific priority areas of national economies.

The third dimension is sustainability, and it was chosen considering it facilitates capturing a country’s capacity (through production and share in consumption) of succeeding in the transition to a new energetic era, governed by avoided or diminished negative effects of pollution (namely emissions) on the environment.

Thus, given the above, performance in the field of bioenergy will be studied through three components: innovation, efficiency, and sustainability.

3.2. Data Collection

The second step refers to selecting indicators that best reflect its components and that are, at the same time, economically, socially, and environmentally relevant. For each component (innovation, efficiency, and sustainability) a set of 10 indicators was formed, based on relevance supported by recent research. The indicators regarding innovation were retrieved mainly from EurObserv’ER online database [47], as it provides an Innovation and Competitiveness section of indicators. The efficiency, and sustainability dimensions were created having as examples similar research works in the energy field [3,4,5,44].

Table 1. Overview of the selected indicators.

No.	Component	Indicator	Unit	Year	Data Source
1	Innovation	1.1 Number of patent families	Number	(2014)	[47]
		1.2 Patent specialization for biofuels	[0, 100]
		1.3 Patents per trillion GDP	Number/trillion GDP
		1.4 Public R&D investment	Euro per capita
		1.5 Share of public R&D investment by GDP	%
		1.6 Private R&D investment	Euro per capita
		1.7 Share of private R&D investment by GDP	%
		1.8 Share of biofuels technologies in total exports	%	(2015)	[47]
		1.9 Net export of technologies for biofuels	Euro per capita
		1.10 Export specialization (biofuels)	[−100, 100]
2	Efficiency	2.1 Generation flexibility in critical hours	%	(2015)	[48]
		2.2 Transmission up-flexibility in critical hours	%
		2.3 Operational flexibility in critical hours	%
		2.4 Turnover in the biofuel field	Euro per capita	(2015)	Based on [49]
		2.5 Bioenergy productivity	Euro per capita/GWh	(2015)	based on [50,51]
		2.6 Bioenergy intensity	GWh/euro per capita		based on [50,51]
		2.7 Share of bioenergy imports in gross inland consumption	%	(2015)	[52]
		2.8 Liquid biofuels % of achievement (2015–2020)	%
		2.9 Biomass % of achievement (2015–2020)	%
		2.10 Bioelectricity % of achievement (2015–2020)	%
3	Sustainability	3.1% of bioenergy in total RES (final consumption)	%	(2015)	[52]
		3.2% of bioenergy in total energy (final consumption)	%
		3.3. Bioenergy per capita	toe per capita	(2015)	based on [50,51]
		3.4 GHG emissions evolution	MtCO2eq	(2017 vs. 1990)	[53]
		3.5 Avoided emissions through bioenergy consumption	MtCO2eq per capita	(2015)	[54]
		3.6 Emissions generated through bioenergy consumption	MtCO2eq per capita
		3.7 Employment in the bioenergy field	Number of jobs per capita	(2015)	[55]
		3.8 Bioelectricity (% in final use in all sectors)	%	(2015)	based on [52]
		3.9 Bioenergy for transport (% in final use in all sectors)	%
		3.10 Bioenergy for heat (% in final use in all sectors)	%

Source: based on mentioned sources.

shows the three sets of indicators, together with measurement units, the sources of the data and the year for which they are reported. Most of the data show values of indicators for 2015, apart from indicators 1.1–1.7, for which the last reported data were at the level of 2014. Also, indicator 3.4, on the evolution of greenhouse gas emissions, presents a comparison between the level of 2017 and that of 1990.

3.3. Data Processing

This step involves the imputation of missing data and our own calculations for obtaining needed indicators. Non available data were reported for Romania and Slovenia for indicator 1.6 and for Slovenia and Slovakia for indicator 1.7, as well as for Italy in the case of indicator 2.1 and for Romania and for Finland in the case of indicator 2.3. Non available data were assigned zero values. Moreover, some indicators are the result of our own calculations based on selected time series (the case of 2.5 and 2.6 indicators for “efficiency” component and 3.3, 3.8–3.10 indicators for “sustainability” component).

3.4. Normalization

As shown in Talukder et al. [41], a variety of normalization techniques are available in the literature, and among them one of the most widely known is min-max normalization. The popularity of the technique made possible the construction of Human Development Index. It has the advantage of rescaling all values of indicators in the [0, 1] range, using zero as a minimum (for the worst reported level of an indicator) and 1 as a maximum (for the best reported level of an indicator). It also takes into account the criteria followed for an indicator. For instance, for almost all indicators, the highest level is reflected by the highest value. However, for 2.6 Bioenergy intensity, the situation is the opposite. As an indicator, it expresses the bioenergy consumption for a unit of GDP per capita, or the cost of converting bioenergy to GDP. So, lower values will be rescaled closer to one, while higher values will be closer to zero. The same happens for indicators 2.7 Share of bioenergy imports in gross inland consumption, 3.4 GHG emissions evolution, and 3.6. Emissions generated through bioenergy consumption.

Given the above explanations, to clarify how normalized values are attributed, the following example appears, for indicator 1.1 Number of patent families. The highest value is registered for Germany, so this country receives 1 as a normalized value, while the lowest value is recorded for Austria, which means it gets 0 as a normalized value. All other normalized values are calculated by using the formula:

$$na=\frac{a_x-\min \left(a_i\right)}{\max \left(a_i\right)-\min \left(a_i\right)}$$

(1)

where na = normalized value; a_x = the value for a country; a_i = all values; min(a_i) = the minimum value among all 16 values; max(a_i) = the maximum value among all 16 values.

The above formula is valid for indicators for which lower values are scaled closer to zero. A second formula was used for indicators for which higher values were rescaled closer to zero (the case discussed above for bioenergy intensity and the other three indicators):

$$na=\frac{\max \left(a_i\right)-a_x}{\max \left(a_i\right)-\min \left(a_i\right)}$$

(2)

3.5. Weighting

In this phase, all normalized values receive a coefficient of importance, which reveals each indicator’s contribution to the overall score of innovation, of efficiency, and of sustainability. Among different type of weighting techniques [56], equal weighting was chosen in this study for simplicity and transparency. So, each indicator accounts for 10% in the level of a component. All resulted values range between 0 and 1 as a consequence of using max-min normalized data.

After establishing a final score (of innovation, efficiency, and sustainability) for a country, a color was assigned to the score, according to the described level fitting one of the five ranges.

3.6. Aggregation

To carry out aggregation, a closer look to the values of the components is needed. All three levels obtained in the previous step have positive values, so they are suitable for geometric mean aggregation.

So, the performance level is obtained by applying the following formula:

$$performance=\sqrt[3]{innovation\times efficiency\times sustainability}$$

(3)

The product of the three dimensions (innovation, efficiency, and sustainability) gathering normalized weighted indicators can take into account the achievement across dimensions. In this way, a low level in one of the three dimensions will be reflected in the overall score of performance and will not be compensated with a suitable surplus in any other dimension [41,56].

3.7. Robustness

The necessity of examining the robustness or the reliability of the new obtained index for performance in the bioenergy field lies in the number and diversity of modelling assumptions used. According to Saisana and Norlen [57], the modelling assumptions are subject to this last phase, so creating the scenarios to test and validate the new index should focus on them: (1) setting up the structure of the analysis framework; (2) choosing the indicators to create the three dimensions; (3) deciding how to impute missing data (4) choosing the normalization technique; (5) deciding on the weights of each indicator; and (6) deciding on how to carry out the aggregation. Within this paper we consider three scenarios to assess rank robustness. This assessment is similar to a sensitivity analysis, as it creates scenarios where a modelling assumption changes while the other remain the same and then it reveals changes or shifts in the overall performance index. The first scenario assumes that the framework does not contain the indicators for which missing data were imputed by us. The second scenario presumes the arithmetic mean for aggregation instead of geometric mean. The third scenario refers also to the indicators, removing from the framework only those indicators which have the overall group normalized values higher than the mean. Also, a fourth scenario was proposed as an alternative (changing the max-min normalization technique with proportionate normalization or with Z-score normalization). Both new methods provided negative values, so the [0, 1] range for the index would be impossible to obtain. In the end, the fourth scenario was abandoned.

The robustness assessment implies recalculating the performance index under the new created scenarios and revealing changes in the performance ranking according to the new obtained values. The changes or shifts are either positive (when the performance index has a new higher value within a scenario as compared to the initial situation determining a country to occupy an upper position in the ranking), or negative (when a country registers a lower performance index within a scenario as compared to the initial case and in accordance, it occupies a lower position in the ranking).

4. Results and Discussion

The following tables include real and normalized values for each indicator and country, and the overall group normalized value for each indicator and the scores obtained after weighting the normalized values for each of the three dimensions. The associated colors are also suggestive, visually facilitating the framing on a recorded level of a certain range (in accordance with

Table 2. Color coding for overall levels of each component.

No.	Color for Describing Level	Range	Described Level
1		[0.8, 1]	Very high
2		[0.6, 0.8)	High
3		[0.4, 0.6)	Average
4		[0.2, 0.4)	Low
5		[0, 0.2)	Very low

Source: own conception.

color coding). In none of the three conducted analyses, was found a score indicating a very high level for a specific dimension. As can be seen, most countries recorded values that allowed them to fall into the category of low or medium results.

Based on the information from the

Table 3. Level of innovation in the bioenergy field.

	Indicators	1.1	1.2	1.3	1.4	1.5	1.6	1.7	1.8	1.9	1.10	Level of Innovation
Country
Austria		1	0.1	2.2	1.09	0.0028	0.13	0.0085	1.36	6.87	39
		0.000	0.000	0.000	0.298	0.252	0.007	0.213	0.124	0.622	0.716	0.223
Czech Republic		3	3.5	15.5	0.07	0.0011	0.92	0.0196	0.55	−0.38	−52
		0.042	0.309	0.223	0.046	0.099	0.052	0.491	0.050	0.424	0.247	0.198
Denmark		16	2.4	61.8	3.34	0.0071	17.85	0.0399	0.02	−15.90	−100
		0.313	0.209	1.000	0.909	0.640	1	1.000	0.002	0	0.000	0.507
Finland		11	2.1	56.3	3.67	0.0098	6.40	0.006	0	0.00	0
		0.208	0.182	0.908	1	0.883	0.358	0.150	0.000	0.434	0.515	0.464
France		33	1.2	16.1	1.38	0.0043	1.31	0.002	9.8	7.46	82
		0.667	0.100	0.233	0.375	0.387	0.073	0.050	0.892	0.638	0.938	0.435
Germany		49	0.5	17.7	0.42	0.0012	1.96	0.0058	3.27	−5.42	−73
		1.000	0.036	0.260	0.114	0.108	0.110	0.145	0.298	0.286	0.139	0.250
Great Britain		11	0.8	5.3	0.38	0.0011	0.62	0.0188	3.03	−4.01	5
		0.208	0.064	0.052	0.102	0.099	0.035	0.471	0.276	0.325	0.541	0.217
Hungary		2	9.5	19	0	0	8.9	0.0008	3.69	204	94
		0.021	0.855	0.282	0.000	0.000	0.051	0.020	0.336	1	1.000	0.356
Italy		6	0.9	3.7	0	0	0.49	0.003	0.88	−1.00	−82
		0.104	0.073	0.025	0.000	0.000	0.027	0.075	0.080	0.407	0.093	0.088
Netherlands		12	1.3	18.2	1.23	0.0031	36.2	0.0042	10.99	136	87
		0.229	0.109	0.268	0.335	0.279	0.120	0.105	1.000	0.654	0.964	0.406
Poland		11	2.9	26.6	0.17	0.0016	0.32	0.0019	0.61	−1.37	−59
		0.208	0.255	0.409	0.046	0.144	0.018	0.048	0.056	0.397	0.211	0.179
Romania		3	3.5	21.6	0.07	0.0009	0	0.0123	0.16	−2.21	−69
		0.042	0.309	0.326	0.018	0.081	0.000	0.308	0.015	0.374	0.160	0.163
Slovakia		2	11.1	27.2	1.55	0.0111	1.64	0	0.89	8.67	57
		0.021	1.000	0.419	0.422	1.000	0.092	0.000	0.081	0.671	0.809	0.452
Slovenia		1	2.5	27.7	0	0	0	0	0.01	−1.94	−100
		0.000	0.218	0.428	0.000	0.000	0.000	0.000	0.001	0.381	0.000	0.103
Spain		8	1.5	8.1	0.21	0.001	0.19	0.0056	1.92	1.03	11
		0.146	0.127	0.099	0.058	0.090	0.010	0.140	0.175	0.463	0.572	0.188
Sweden		3	0.4	7.1	3.25	0.0073	1.16	0.0029	1.62	−11.70	55
		0.042	0.027	0.082	0.885	0.658	0.065	0.073	0.147	0.115	0.799	0.289
Overall group normalized value		3.250	3.873	5.015	4.609	4.721	2.019	3.291	3.530	7.193	7.706

and Figure 2, one can see that from the innovation point of view all analyzed countries score in the bottom part (very low, low, average) of the range. Looking closer to Figure 2, one can observe that countries separate themselves in three categories: northern and western European countries are situated at a low level regarding innovation in the bioenergy field, central European countries are situated at an average level and eastern and southern European countries are situated at a very low level regarding innovation in the bioenergy field.

The main reasons that can describe this situation are the following:

• lack of sustained investments in the research field regarding bioenergy, private and public investments.
• low number of patents, indicator which can be closely related to lack of resources to understand and to study the bioenergy activity sector.
• These particular reasons are also supported by the low values recorded in the overall normalized values for the group of countries. The cumulative amount has a maximum of 16 points, given by the maximum of 1 (as normalized value) that can be reached by each of the 16 studied countries. From all ten indicators composing the innovation dimension, only two managed to get closer to the mean, slightly reaching 7.193 (indicator 1.9) and 7.706 (indicator 1.10) as overall normalized values for the group of countries.

Moving on to the second dimension, Efficiency, it is widely known that it represents the ability to achieve a desired level of performance by using the least number of inputs in order to obtain the highest amount of output [58]. Based on information given in

Table 4. Level of efficiency in the bioenergy field.

	Indicators	2.1	2.2	2.3	2.4	2.5	2.6	2.7	2.8	2.9	2.10	Level of Efficiency
Country
Austria		25	65	44	264.417	0.70	1.44	7.1	132	108	86
		0.309	0.656	0.454	0.306	0.231	0.852	0.368	0.936	0.485	0.723	0.532
Czech Republic		42	48	72	124.3	0.39	2.56	−0.2	48	111	107
		0.519	0.479	0.742	0.115	0.105	0.694	0.462	0.264	0.515	0.947	0.484
Denmark		81	35	5	199.65	1.46	0.69	35.7	89	100	48
		1.000	0.344	0.052	0.218	0.538	0.958	0.000	0.592	0.408	0.319	0.443
Finland		47%	70%	0	773.06	0.41	2.44	1.1	89	101	88
		0.580	0.708	0.000	1	0.113	0.711	0.445	0.592	0.417	0.745	0.531
France		50	14	34	102.01	0.26	3.90	3	84	58	34
		0.617	0.125	0.351	0.085	0.053	0.505	0.421	0.552	0.000	0.170	0.288
Germany		25	19	7	142.61	0.16	6.36	−2.1	48	107	102
		0.309	0.177	0.072	0.140	0.012	0.158	0.486	0.264	0.476	0.894	0.299
Great Britain		31	98	8	45.02	0.42	2.39	28.8	22	72	112
		0.383	1.000	0.082	0.007	0.117	0.718	0.089	0.056	0.136	1.000	0.359
Hungary		27	50	38	103.49	0.38	2.64	−7.8	35	161	65
		0.333	0.500	0.392	0.087	0.101	0.683	0.560	0.160	1.000	0.500	0.432
Italy		0	63	97	60.53	0.22	4.56	20	47	137	103
		0.000	0.635	1.000	0.028	0.036	0.412	0.202	0.256	0.767	0.904	0.424
Netherlands		14	44	38	39.64	2.29	0.44	−42	36	124	30
		0.173	0.438	0.392	0	0.874	0.993	1.000	0.168	0.641	0.128	0.481
Poland		56	54	56	69.72	0.13	7.48	4.6	41	97	70
		0.691	0.542	0.577	0.041	0.000	0.000	0.400	0.208	0.379	0.553	0.339
Romania		23	2	0	49.31	0.20	5.11	1.1	41	87	18
		0.284	0.000	0.000	0.013	0.028	0.334	0.445	0.208	0.282	0.000	0.159
Slovakia		29	63	18	132.8	1.21	0.83	−1.3	78	89	97
		0.358	0.635	0.186	0.127	0.437	0.938	0.476	0.504	0.301	0.840	0.480
Slovenia		36	60	16	63.01	2.60	0.39	4.8	15	109	40
		0.444	0.604	0.165	0.032	1.000	1.000	0.398	0.000	0.495	0.234	0.437
Spain		37	23	34	43.91	0.35	2.83	−5.9	27	81	58
		0.457	0.219	0.351	0.006	0.089	0.656	0.535	0.096	0.223	0.426	0.306
Sweden		9	31	34	4690	0.42	2.36	7.3	140	89	65
		0.111	0.302	0.351	0.602	0.117	0.722	0.366	1.000	0.301	0.500	0.437
Overall group normalized value		6.568	7.365	5.165	2.809	3.854	10.333	6.654	5.856	6.825	8.883

, we managed to create a map revealing the distribution of countries according to their efficiency level in Figure 3.

The results show that the weakest score value of 0.166 belongs to Romania. Northern and southern countries register an average level of efficiency in the bioenergy field, while countries situated partially in Central and Western Europe register a low level of efficiency in the bioenergy field. The situation can be attributed to the following reasons:

• For several countries, the data show a lack of flexibility regarding the system utilization.
• The turnover in the biofuel field is situated at a low competitive level, with implications for other areas.

However, the overall normalized values calculated in Table 4 show that for indicator 2.6 Bioenergy intensity, the group of countries reaches a 10.333 out of 16. This indicates that the majority of countries deals with the cost of converting bioenergy in GDP.

As Figure 4 shows, countries from northern Europe have a high level of sustainability in the bioenergy field, above average, while red represented countries have a low level of sustainability in the bioenergy field, with the mention that values are not very different from one country to another, as reported in

Table 5. Level of sustainability in the bioenergy field.

	Indicators	3.1	3.2	3.3	3.4	3.5	3.6	3.7	3.8	3.9	3.10	Level of Sustainability
Country
Austria		52	17.8	0.57	5	0.37	0.01	1444.39	13.16	7.72	18.08
		0.248	0.465	0.348	0.151	0.206	0.681	0.260	0.471	0.166	0.406	0.340
Czech Republic		83.9	13.7	0.33	−69	0.44	0.01	2106.61	8.92	12.38	6.16
		0.842	0.329	0.174	0.340	0.249	0.740	0.410	0.237	0.285	0.138	0.374
Denmark		70.8	22.8	0.5	−21	0.65	0.02	1166.14	7.16	11.18	44.53
		0.598	0.631	0.297	0.217	0.379	0.517	0.197	0.140	0.254	1.000	0.423
Finland		82.5	33.9	1.47	−15	1.70	0.04	4715.13	6.11	12.06	21.73
		0.816	1.000	1.000	0.202	1.000	0.000	1.000	0.082	0.277	0.488	0.586
France		60.8	8.9	0.17	−75	0.07	0.00	953.98	22.74	3.92	8.09
		0.412	0.169	0.058	0.355	0.029	0.959	0.149	1.000	0.069	0.182	0.338
Germany		58.3	8.9	0.25	−327	0.55	0.01	1230.33	13.45	22.68	8.03
		0.365	0.169	0.116	1.000	0.313	0.697	0.211	0.487	0.548	0.180	0.409
Great Britain		54.7	4.8	0.13	−304	0.34	0.01	366.36	14.90	40.36	0.30
		0.298	0.033	0.029	0.941	0.192	0.789	0.016	0.567	1.000	0.007	0.387
Hungary		92.4	13.8	0.26	−30	0.16	0.00	2648.25	7.24	7.70	4.97
		1.000	0.332	0.123	0.240	0.084	1.000	0.532	0.144	0.165	0.112	0.373
Italy		56.4	9.2	0.18	−83	0.20	0.00	705.64	10.99	15.71	8.35
		0.330	0.179	0.065	0.376	0.108	0.865	0.093	0.351	0.370	0.188	0.292
Netherlands		68.5	3.8	0.09	−21	0.18	0.01	295.85	15.80	22.55	16.60
		0.555	0.000	0.000	0.217	0.094	0.838	0.000	0.617	0.545	0.373	0.324
Poland		83.9	10.6	0.19	−59	0.25	0.01	1723.43	11.91	13.04	4.70
		0.842	0.226	0.072	0.315	0.138	0.856	0.323	0.402	0.302	0.106	0.358
Romania		61.2	16.6	0.18	−134	0.02	0.00	1625.51	5.59	1.24	2.01
		0.419	0.425	0.065	0.506	0.000	1.000	0.301	0.053	0.000	0.045	0.281
Slovakia		67.4	8.9	0.19	−30	0.24	0.00	2600.83	15.98	15.87	14.43
		0.534	0.169	0.072	0.240	0.129	1.000	0.522	0.627	0.374	0.324	0.399
Slovenia		57.8	13.2	0.3	−1	0.12	0.00	1114.95	4.63	3.67	5.58
		0.356	0.312	0.152	0.166	0.057	1.000	0.185	0.000	0.062	0.125	0.242
Spain		38.7	6.8	0.12	64	0.09	0.00	792.26	17.46	9.04	0.00
		0.000	0.100	0.022	0.000	0.039	0.941	0.112	0.708	0.199	0.000	0.212
Sweden		55.4	32.6	0.97	−17	0.87	0.02	2800.76	10.55	8.87	28.73
		0.311	0.957	0.638	0.207	0.506	0.439	0.567	0.327	0.195	0.645	0.479
Overall group normalized value		7.924	5.498	3.232	5.476	3.522	12.322	4.878	6.213	4.810	4.318

.

An important point that involves sustainability is regarding avoided emissions through bioenergy consumption, with many countries having difficulties in obtaining a higher score for this indicator. Technologies for obtaining greener biofuels are yet to be developed, and where possible, the use of new generations of biofuels will replace the process of burning biomass, which is in fact the highest contributor in terms of generated emissions.

All things considered, after obtaining levels for all three dimensions, the performance within the bioenergy field can be calculated by using the geometric mean aggregation, explained in the methodology section.

The summary table (

Table 6. Summary of results.

No.	Country	Level of Innovation in the Bioenergy Field	Level of Efficiency in the Bioenergy Field	Level of Sustainability in the Bioenergy Field	Level of Performance in the Bioenergy Field
1	Austria	0.223	0.532	0.340	0.343
2	Denmark	0.507	0.443	0.423	0.456
3	Finland	0.464	0.531	0.586	0.525
4	France	0.435	0.288	0.338	0.349
5	Germany	0.250	0.299	0.409	0.313
6	Italy	0.088	0.424	0.292	0.222
7	Great Britain	0.217	0.359	0.387	0.311
8	Poland	0.179	0.339	0.358	0.279
9	Czech Republic	0.198	0.484	0.374	0.330
10	Romania	0.163	0.159	0.281	0.194
11	Slovakia	0.452	0.480	0.399	0.442
12	Slovenia	0.103	0.437	0.242	0.222
13	Spain	0.188	0.306	0.212	0.230
14	Sweden	0.289	0.437	0.479	0.393
15	Netherlands	0.406	0.481	0.324	0.398
16	Hungary	0.356	0.432	0.373	0.386

Source: own calculations

) shows that the lowest scores were obtained for the level of innovation in the bioenergy field, while the highest scores were found in establishing the level of efficiency. Depending on their scoring, the performers in the field of bioenergy are Finland and Denmark, while the non-performers in the ranking are Italy, Slovenia, and Romania (

Table 7. Country ranking based on the level of performance in the bioenergy field.

No.	Country	Level of Performance
1	Finland	0.525
2	Denmark	0.456
3	Slovakia	0.442
4	Netherlands	0.398
5	Sweden	0.393
6	Hungary	0.386
7	France	0.349
8	Austria	0.343
9	Czech Republic	0.330
10	Germany	0.313
11	Great Britain	0.311
12	Poland	0.279
13	Spain	0.230
14	Slovenia	0.222
15	Italy	0.222
16	Romania	0.194

). In addition, Figure 5 highlights the results obtained in Table 7, showing the countries outperforming in the bioenergy field (blue color), but also the countries which have to continue to develop and apply strategies in the bioenergy field (red color) to perform better.

As we already presented in the methodological part of the paper, similar rankings were developed for the field of renewable energy and one of them refers to the Nexus Index. This index assesses the sustainability of bioenergy production by studying the efficiency of natural resources consumption, namely main crops such as maize. The index is calculated for 2014, and it reveals in top 10 most efficient countries [43], Netherlands (3rd rank), Austria (6th position), Germany (8th position) and Slovenia (9th place). As such, Netherlands is found to be among performers in the bioenergy field by analyzing the Nexus index as well. At the same time, in the present ranking of performance in the bioenergy field, Slovenia joins the non-performers team, scoring low, while in the Nexus index, it (based only on the efficiency of using energy crops) joins the top 10 most efficient countries in terms of bioenergy production. However, our index is developed for 16 countries only, while the Nexus index is calculated for 191 countries.

Figure 6 is a graphical representation of the analyzed countries based on the dominant component of bioenergy performance treated in this paper. By dominant component, we understand the dimension in which a country holds the best score. As we can see, northern Europe and several countries from central Europe hold supremacy in the sustainability dimension, having good performance for indicators included in this category. Represented by yellow, Denmark is the only country from the group of 16 that has a higher value for indicators regarding innovation, being followed by France, while several countries from Central Europe and Sothern Europe hold supremacy for Efficiency.

To check the reliability of the new performance index in the bioenergy field, the rank robustness is tested through a sensitivity analysis based on three scenarios. In this way, the shifts in ranks for the studied countries are capable of revealing how changes in the initial modelling assumptions influence the country ranking.

Table 8. Robustness assessment.

No.	Country	Level of Performance	Shifts in Ranks under Different Scenarios
			(1) Scenario	Shift	(2) Scenario	Shift	(3) Scenario	Shift
1	Finland	0.525	0.564	0	0.527	0	0.526	0
2	Denmark	0.456	0.49	1	0.458	0	0.435	0
3	Slovakia	0.442	0.435	1	0.444	0	0.384	0
4	Netherlands	0.398	0.432	1	0.404	0	0.368	1
5	Sweden	0.393	0.418	1	0.402	0	0.380	−1
6	Hungary	0.386	0.409	−4	0.387	0	0.349	0
7	France	0.349	0.365	1	0.354	1	0.318	0
8	Austria	0.343	0.35	−1	0.365	−1	0.316	0
9	Czech Republic	0.330	0.334	1	0.352	0	0.298	0
10	Germany	0.313	0.316	1	0.319	1	0.284	0
11	Great Britain	0.311	0.311	−2	0.321	−1	0.259	1
12	Poland	0.279	0.274	0	0.292	0	0.268	−1
13	Spain	0.230	0.244	1	0.235	2	0.183	2
14	Slovenia	0.222	0.235	−1	0.261	0	0.185	0
15	Italy	0.222	0.226	0	0.268	−2	0.195	−2
16	Romania	0.194	0.197	0	0.201	0	0.173	0

Source: own calculations.

shows the results of three sensitivity analyses which include:

(1)

The removal from the framework of indicators for which missing data were imputed by us.

(2)

The use of the arithmetic mean for aggregation instead of geometric mean.

(3)

The exclusion from the framework of those indicators which have the overall group normalized values higher than the mean for the group.

The highest number of shifts can be found in the first scenario, so the choice of indicators is crucial for the performance assessment framework. Positive shifts reveal an improvement for a country and its position in the performance ranking. As expected, negative shifts tell that an inferior rank was assigned to a country. The overall robustness of the new created index is satisfactory, as the average shift in rank across all countries is 0 for all scenarios.

5. Conclusions

This research intended to create a performance assessment framework for the bioenergy field in order to provide new information on the current status of development in this field. Studying the weakest results in terms of performance and finding appropriate ways to counteract represents a great challenge for policy makers in establishing future targets to achieve by renewables in the battle of reducing climate change effects. The role of renewables has been worldwide recognized a long time ago, so tackling the concept of performance in this field supports the idea of revealing progress towards a cleaner environment created by a cleaner and greener consumption.

The assessment framework can be applied for other renewable energy sources as well, being easy to construct and use. Its main contribution is providing comprehensive information on the bioenergy field development, enclosing innovation, efficiency, and sustainability as performance components. It can be used as a screening tool for researchers and policy makers even for investors. The ranking based on performance levels facilitates comparative analyses among countries, reveals outperformers and non-performers. The effective measurement and benchmark of the levels in the three dimensions across countries are also important contributions of the present research. The weakest results may constitute warning signs for decision makers, while revealing what leads to them, may be helpful in selecting appropriate measures to increase overall performance of countries.

As for the limitations of this research, the timeliness of data is the major drawback. The availability of data for so many countries, determined us to choose 2015 (in majority of cases) in order to include as many indicators as possible in the framework. However, at a point, the risk of revealing outdated information appears if efforts of supplying up to date data are not made by concerned institutions. Another limitation is given by the choice of indicators to be included within the framework. The sensitivity analysis conducted in the robustness assessment phase showed how the overall performance level could change if removing indicators from the framework. No scenario of adding relevant indicators was used. For instance, the third dimension included in the performance level, sustainability, does not cover all the relevant aspects of the effects on the environment, but only emissions as a consequence of pollution. Other indicators revealing the effects on the environment can be used, such as those related to raw material source and the procurement chain of the biofuels. Furthermore, another limitation regarding indicators refers to not including the component of waste into the bioenergy production, but only solid biomass, biogas and liquid biofuels. For some countries, waste can definitely contribute to the bioenergy production value (through municipal waste and industrial waste categories) and influence the calculation of some indicators, such as: bioenergy intensity, bioenergy productivity, and bioenergy per capita. In the end, we consider that the lather two limitations can constitute, if properly approached, new directions of research and of developing the proposed performance framework.