A Nash Equilibrium Approach to the Brazilian Seasonalization of Energy Certificates

Abstract

This paper presents a Nash equilibrium approach to model a non-cooperative game that takes place among Brazilian hydro-generating companies in the annual process called “seasonalization”. We have “remade” the seasonalizations that occurred in the period of 2013–2020 by using our model and comparing the financial outcomes with the current ones that resulted from the seasonalizations the generators made each year, with financial improvements in almost every year. By using the Nash equilibrium approach, it is possible to achieve optimal decisions concerning the seasonalization process that were not evident by using traditional methods. This approach is useful for companies willing to enhance their income and to improve their risk management by making better choices in seasonalization.

1. Introduction

The reform of the electricity markets around the world in the 1990s and the introduction of competition in the energy transactions provided new subjects for the application of game theory. The necessity of making better decisions in a competitive environment forced energy companies to not only strengthen their quantitative and fundamentalist analysis capabilities, but also to explore the use of strategic games theory in order to model their competitors’ behaviors.

Given the vastness of the energy industry, many applications of game theory were developed to tackle different problems. A Nash equilibrium was proposed in [1] to maximize the expected profit for the generators, when spot prices have already been announced on the Australian electricity market. In order to find the Nash equilibrium, Ref. [2] implemented four numerical methods and tested them in four experiments of non-cooperative games, including an electricity market game.

Another field of application towards the Nash equilibrium approach is regarding the market pricing and bid definition considering the physical system and the interaction among the participant agents. A Nash/Stackelberg equilibrium solution to find the market price and then define the optimal consumption of the consumer was proposed in [3], where the operator and consumer payoff curves were analytically obtained, and the proposed algorithm found the intersection between them as the Nash equilibrium of the strategic game.

A Nash equilibrium approach was proposed in [4] to help with the interaction between the ISO and generators agents in the generation maintenance schedule problem. The generator maintenance process was modeled as a non-cooperative dynamic game, in order to model their preferences and analyze the interaction among them. The Nash equilibrium was found using the technique of backward induction among the decision trees.

In order to find the suppliers’ optimal strategies in a deregulated electricity market, Ref. [5] proposed a co-evolutionary algorithm to find the Nash equilibrium, by modeling the market as a two-settlement market. A vast literature survey on how the bidding strategies solutions have been addressed and solved in the literature was presented in [6], by pointing out that Nash equilibrium may be used to evaluate different situations.

The Brazilian energy market has also gone through the deregulation process and was opened to competition in 2001. Although regulation does not allow residential clients to take part in the free market, the load that is represented by large industrial and commercial customers is quite relevant, and today about 30% of all energy consumed in the country is freely traded. However, the market deregulation kept some structures from the preexisting electrical system coordination, such as the price formation through computational calculations [7,8,9], which are centrally performed by the ISO and the CCEE. Thus, there is no field for the game theory application towards price formation on a short-term period—although, this theory can be applied to the pricing of bilateral contracts. Nonetheless, game theory has been used to help policy makers to evaluate the impacts of changing the market design of Brazilian power market, Ref. [10] proposed a Cournot–Nash model to evaluate the implementation feasibility of bid-based pricing in their market, suggesting that this change is possible.

Considering the predominance of hydro generation in the Brazilian power system, there is a very important regulatory structure that works as a collective risk management mechanism for the hydraulic generators. It is the ERM, and all hydro generators centrally dispatched must take part in it. This structure is perfect for the use of game theory because the ERM determines the resources that will be available for each of the participants in the settlement of short-term positions, based in their decisions. This is so because each hydro generator is not entitled to use the energy they produce to confront the contracts they sell in the market but, instead of that, they receive a share of the energy generated by all of the participants in the mechanism.

The fraction each generator received is calculated proportionally to the PG of the plant at a given month. Furthermore, in their turn, these monthly quotas of PG are defined in an annual process called seasonalization of PG. So it means that seasonalization is merely the distribution of an annual amount of PG into monthly amounts whose sum equals the original annual value, and, at the end of every year, all hydro-generators participates in this process by allocating their power plants’ PG for the next year.

This happens because the strategic game emerges in seasonalization due to the dependence of the results of each agent on other agents’ decisions. Considering that each power plant is entitled to have a share of the energy available to the ERM as a whole, each power plant is dependent on how the other participants have made their seasonalization. The subject of seasonalization has drawn the attention of several researchers, who have presented some different methods to deal with the problem and improve the decisions made by the generators.

An optimization model to help decision makers to define their seasonalization strategy with the assessment of financial risk and return was proposed in [11]. However, this work did not take into account the decision of the other ERM participants. Another approach to help in the decision making is a simplified model that replicated the short-term settlement in the Brazilian market of one agent and was proposed in [12,13]. Based on that model, a multi-objective problem (return/risk) was created to propose a seasonalization profile for this agent, considering a predefined allocation of the other ERM agents. A deterministic optimization method was used to solve the problem in [12] and a genetic algorithm was used to solve the problem in [13].

The works that are mentioned above did not use any kind of interaction modeling to define the best strategy for the agents or assess the impact of their decisions on the financial results of the mechanism at all.

The game theory applied in the seasonalization process was studied by [14], where an extensive study was made by comparing the different types of games (cooperative and non-cooperative) and its possible solutions. The problem was designed considering one agent versus the all other agents, considered as one big agent. In the end, the methodology considered the Nash equilibrium to propose a seasonalization profile for both players and a set of optimal solutions proposed to help assess the agent’s decision.

A framework was designed and proposed in [15]. This framework was able to analyze different possibilities of seasonalization profile for the ERM by using a market intelligence tool, and a Nash Equilibrium approach was used as well in order to propose a seasonalization profile to help assess the agent’s decision.

The game theory approaches proposed previously considered only a game with two agents, one generation agent against the other agents of the ERM. In order to measure how the decision of one agent may affect the result of the mechanism, in the present paper, we intend to model this interaction by proposing a non-cooperative game approach for the seasonalization process with more than two agents. We have modeled four agents according to the four price zones in Brazilian market in order to catch up the particularities of each price zone that may induce the decisions of the agents, something that has not been studied in the literature so far. Moreover, a Nash equilibrium approach was used to solve this non-cooperative game among the four agents, and it provides a new path to evaluate seasonalization strategies and, therefore, allows the agents to make better choices to enhance their financial outcome and their risk management.

2. The Short-Term Settlement and Hydro Hedge Mechanism

The hydroelectric power plants have an intrinsic variability in generating energy due to the variability of the inflow of the rivers to the dams. So, for every hydropower plant, there is always a risk of not being able to generate up to the level of the long-term expected energy, if the inflows are below historical average. As the Brazilian power system is interconnected in almost all of the country, encompassing different geographical areas with different hydrological regimes, a compulsory mechanism was created to share the lack of inflow in a certain area, that might be compensated with high inflow in another area. This mechanism is called the energy reallocation mechanism (ERM) [16].

This is an important hedging mechanism through which plants located in areas with transient low inflows receive energy generated by plants that are located in other areas with transient high inflows. So, according to this, ERM provides a more constant power output for all the plants that take part in it.

As mentioned above, the fraction of energy that each plant is entitled to monthly in the ERM is calculated proportionally to the PG of the plant at a given month.

The ratio between the ERM total production and the sum of the PG of its participants is called GSF, and it is used monthly to allocate the energy produced by the set of ERM plants among these plants. So, this factor ($\eta$) is given by (1):

$${\eta }_m=\frac{{\displaystyle \sum _{n\in N}}{h}_{n,m}}{{\displaystyle \sum _{n\in N}}{g}_{n,m}}$$

(1)

where m represents the month and n represents the plant $n\in N$ that participates in the ERM, h is the hydro-generation of a given plant n and g is the monthly seasonalized PG of a given plant n. This GSF factor ($\eta$) adjusts the individual PG of all plants every month. This means that the amount of energy available for the monthly short-term settlement of each agent will be adjusted using this factor.

The Brazilian short-term settlement is a monthly event and an ex-post-process that happens after the generation and the consumption of the system occurred, the imbalance between the requirements and resources of each agent on each zone is then settled considering the zonal spot prices published by CCEE. The main role regarding CCEE is to register and process the volume of all the energy contracted, consumed, and generated in the Brazilian electricity market, following the market rules defined by the ANEEL. The market rules that define the short-term settlement are defined in [17] and it was used, in a simplified way, to model the short-term settlement. The PG, its adjustments, and the operation rules of the ERM are described in [18,19]. Furthermore, the use of those references are the base of the mathematical model and the optimization problem explored in this paper. Additionally, CCEE computes the spot price (the spot price in the Brazilian electricity market is called PLD) using the information from the ISO.

3. The Price Zones

The Brazilian electricity market has four price zones in which the participants of the market may trade energy by selling or buying contracts in each zone, assuming the risk of different spot prices among the zones. The zones defined in the Brazilian market are the Southeast (SE), South (S), Northeast (NE), and North (N) zones. Hence, the ISO calculates the marginal cost of each zone by using optimization models to define the plants operation [20] and passes that information to the CCEE that calculates and publishes the PLD for each of these four zones, that are used to calculate the short-term exposition [21].

The short-term exposition ($R_{a,m}$) of each market agent $a\in A$ is calculated for every zone $z\in Z$ that this agent has contracts (purchase or sale contracts) or PG. A simplification of the short-term market clearing is given by (2):

$$R_{a,m}=\sum _{z=1}^4(B_{a,z,m}-C_{a,z,m})\times P_{z,m}$$

(2)

where a, z, and m represents the agent, the zone, and the month, respectively, B are the agent resources (buying contracts and adjusted physical guarantee), C the requirements (selling contracts and consumption), and P is the spot price.The agent resources B are given by (3):

$$B_{a,z,m}=\left(b_{a,z,m}\right)+(G_{a,z,m}\times {\eta }_m)$$

(3)

where b are buying contracts and $G_{a,z,m}$ are the total amount of seasonalized PG.

4. The Seasonalization Process

As pointed out previously, every hydro-generator has an annual amount of energy certificate, its PG, which is determined by the ANEEL by the time the concession to build the power plant is granted. The PG is the amount of energy each power plant is allowed to sell in the market. Since every generation is commanded by the ISO, no generator is able to determine what their plants are going to produce. However, this is not a problem, because they sell their PG, not the power they produce locally at their plants.

When a generator goes into the market to sell energy, it is selling part of its physical guarantee. The consumers need to have some flexibility on how to use the annual amount of energy they buy, and that is why contracts usually have clauses that allow the consumers to seasonalize that annual amount, in order to distribute the annual amount into monthly quantities that are suitable for their consumption in the following year.

So, the generators need to perform their own seasonalization of PG to match the requirements of their contracts. This process of seasonalization consists of distributing, throughout the year, the PG of each ERM participating plant. Thereafter, each plant/player has each month the right to a fraction of the energy produced by the ERM in the month, this fraction is given by the ratio between the seasonalized PG in the month and the sum of the PG of the entire ERM plants that month, described in (1).

The seasonalization process has two different phases. Both phases occur prior to the start of the year, so the decisions of the players are made without knowing how the operation of the system—and consequently, the spot prices—will be. First, the load has to inform for their sellers their monthly needs, usually until the end of November, and after that, the participants of the ERM have to seasonalize their own PG, usually until the end of the first half of December.

For both the load and the hydro-generator, this is a simple process, and may be used to hedge their energetic position. The load follows their consumption profile, allocating more energy where its demand is higher and the generator follows the load profile of the sold seasonalized contracts. That means they allocate more energy where they have to deliver more energy.

However, this strategy of PG allocation is not mandatory, and the players may simply try to predict how the system’s operation will be in the next year and, as a result, the spot prices, and allocate energy trying to improve their financial results. At this point, it is important to remember that the decision that one agent makes interferes with the results of the decisions of all agents who are taking part in the ERM, because the more energy is allocated in a given month by all agents, the smaller the share that each agent will receive. Figure 1 shows some possible strategies for physical guarantee seasonalization.

Until the seasonalization process of 2012, the majority of the players tried to hedge their energetic position. However, at the end of 2012, some regulatory changes allowed for more aggressive seasonalization strategies, and players started to allocate a lot of energy on months that they anticipated that the spot prices would be higher. From that year on, the agents started to allocate more and more energy in the second half of the year, leading to high imbalances between the expectation of having more energy available to the monthly settlement in that part of the year, and the reality of the non-existence of physical generation to stand against all PG allocated. Figure 2 shows the evolution of PG seasonalization compared to the load profile.

As described in (3), B represents two kinds of resources for each agent: the energy contracts purchased and the PG, which is is adjusted monthly considering the level of seasonalization that the agent made for that specific month, the level of energy generated by all the hydro-generators within the ERM, and the level of seasonalization that all other agents made accordingly to their own strategies. So, the game emerges from the fact that the decision that each generator makes composes the relation that will be applied to all generators, as described in (1).

Nonetheless, at the end of the day, the amount of energy that every player will get is proportional to the seasonalized PG, and is also proportional to the ratio between the total PG allocated and the total hydro-generation of a given month. So, for a given amount of physical generation in the system, increasing the energy that a generator allocates to a specific month during the seasonalization process will not necessarily be able to increase the energy it will receive from ERM because everybody else is doing the same thing—that is, increasing their allocation for that month.

This kind of “herd movement” may be due to the (justified) perception that prices tend to be higher in the second half of the year, but also due to the fear of doing things differently from the other participants. In other words, participants may have the perception that, whenever all of them concentrate their seasonalization in a certain period of the year, all of them are going to be negatively affected, because GSF is going to decrease. This strategy may be a good one if the relation between the spot price and the hydro-generation of a given period is much higher than in another one. However, due to the lack of means to evaluate—and demonstrate—the benefits of other alternatives, doing what everybody else does is a good choice. This behavior may be explained by the fear of missing out paradigm: agents choose to imitate the expected actions of their cohort to avoid suffering losses [22].

Without this global financial assessing, this agent is lured to allocate more energy where its own result is maximized, and every other agent do this way as well, inducing a herding incentive. Moreover, this allocation strategy made by each player affects directly the GSF, that is, the result of what is essentially a game among these players will define the final result of the financial settlement of the entire mechanism.

Considering the strategic game that this problem represents, we propose a game equilibrium model to evaluate how the decision of all the agents should be, considering a given spot price prediction scenario, and what the financial result for the mechanism as a whole could be.

5. Applying Nash Equilibrium on the Seasonalization Game

A game may be defined as a problem that must consider the possible actions of all agents involved. Those actions must be interdependent, and none of the decision makers has any control of the outcome of this problem.

For non-cooperative games, the Nash equilibrium [23] is an approach used to solve and find a solution that satisfies all the optimization conditions for each player, given the choice of all the other players.

The Nash equilibrium may be represented as several optimization problems that are simultaneous and dependent.

$$\begin{array}{cc}& x_1^{\ast }=\underset{x_1}{arg\; min}{f}_1(x_1,x_2^{\ast },\dots ,x_n^{\ast }):(x_1,x_2^{\ast },\dots ,x_n^{\ast })\in \mathbb{F}\\ \hfill & \hfill x_2^{\ast }=\underset{x_2}{arg\; min}{f}_2(x_1^{\ast },x_2,\dots ,x_n^{\ast }):(x_1^{\ast },x_2,\dots ,x_n^{\ast })\in \mathbb{F}\hfill \\ \hfill & ⋮\hfill \\ \hfill & \hfill x_n^{\ast }=\underset{x_n}{arg\; min}{f}_n(x_1^{\ast },x_2^{\ast },\dots ,x_n):(x_1^{\ast },x_2^{\ast },\dots ,x_n)\in \mathbb{F}\hfill \\ \hfill \end{array}$$

(4)

The seasonalization problem is a kind of game in which the allocation decision of each ERM participant directly interferes in the results of all the participants of the mechanism, once the ratio of the system generation that every participant will receive depends on the total allocation for a given month, as shown in (1).

The seasonalization problem is a strategic game, but it is also a problem of decision making under uncertainty. This is due to the stochastic characteristic of hydro-generation and spot prices. So, the solution must consider different scenarios of inflows to the hydropower plants in the system. The ISO operation planning is derived as a part of several optimization models that optimizes the system for a long run base. As a result of this optimization, several hydrological, operation, and spot prices scenarios are generated. Figure 3 and Figure 4 shows the scenarios projected in December of 2019–2020.

As we can see from Figure 3 and Figure 4, the dispersion of scenarios is very wide and may be used to guide the decisions regarding the seasonalization, in order to assess risk and return from those projections. Nevertheless, in this work, we have based the decision to find the Nash equilibrium by using only the average of the projected scenarios.

As we can see from Figure 5, the seasonalization decisions from the agents seem to be anchored on the spot prices of the last year and the expectation that the spot prices of the dry season in the next year would be much higher than those of the wet season and not in average forecast for the next year that the ISO made available.

In order to check whether this was a wise decision, we formulated a Nash equilibrium model with four agents (representing one agent at each price zone). The objective function of each player will represent its annual short-term exposition given by (2).

Every player has a monthly load profile to supply and has an annual energy surplus of 10% of their hydro resources, given by a general hedge strategy, the decision variables are how much energy (in MWh) should be allocated in each month, given some upper and lower bounds ($\overline{X}$ and $\underline{X}$). To evaluate the players’ decisions, we used the average of the scenarios of hydro-generation and spot prices, based on the ISO projection for the next year.

An equilibrium model based on Nash equilibrium was used to propose a new seasonalization profile for each of the four players. The objective function of each agent a is given by:

$$x_a^{\ast }=\underset{x_a}{arg\; min}{f}_a(x_a,x_{-a}^{\ast }):(x_a,x_{-a}^{\ast })\in \mathbb{F}$$

(5)

where $x_a\in {\mathbb{R}}^{12}$.

The feasible set ($\mathbb{F}$) is given by the following:

$$\mathbb{F}=\{\left(x_a\right)|\sum _{m=1}^{12}{x}_{a,m}=X_a,{\underline{X}}_a\le x_{a,m}\le {\overline{X}}_a\}$$

(6)

where X is the total amount of energy that may be seasonalized and ${\underline{X}}_a$ is the minimum amount and ${\overline{X}}_a$ is the maximum and amount of energy to be allocated in a given month.

The objective function for each player is to maximize the short-term financial result and is given by the following:

$$\begin{array}{cc}\hfill & \hfill f_a(x_a,x_{-a}^{\ast })=\sum _{m=1}^{12}\left(\sum _{z=1}^4(C_{a,z,m}\times P_{z,m})-\sum _{z=1}^4(\frac{{\sum }_{n=1}^4{h}_{n,m}}{x_{a,m}+x_{-a,m}^{\ast }}\times x_{a,m}\times P_{z,m})\right)\hfill \\ \hfill \end{array}$$

(7)

Note that for this work, we consider that each agent has resources and sales contracts in only one zone price.

As the seasonalization process is the monthly allocation of the PG over the months, there is only one limit applicable to this agent decision, such that it may not allocate more energy than its plant is capable of generating in a given month, as the plant’s installed capacity.

However, in this work, we used as constraints the maximum and minimum allocation occurred in the period of analysis (from 2013 to 2020) that was $\underline{X}=50\%$ and $\overline{X}=160\%$ of the annual defined PG.

The Nash equilibrium was achieved using the relaxation algorithm first proposed in [24] and improved in [25], which in this paper we used 0.5 as fixed step size and 1000 iterations. To test the robustness and the scalability of the algorithm, we made some tests changing the number of agents from 2 to 10, that showed a linear growth of function evaluations because of the gradient evaluation is proportional to the number of agents A ($13\times N_a$) as shown in Figure 6.

Notice on Figure 7 that 1000 iterations were enough to converge the problem with 5 agents to the equilibrium up to machine epsilon, considering the maximum average standard deviation among the decision variables—from this point on, the errors increase.

6. Results

The objective of this paper was to analyze the behavior of market agents and compare it with a strategy calculated by a simulation model with a game equilibrium (Nash equilibrium) approach. In order to evaluate the results, we consider the mechanism effect as a whole, so we are not observing the results of a specific agent.

Once the model was developed, we have made several optimizations changing the starting point (the starting point must always be a feasible point), and those changes did not alter the output, indicating that, empirically, equilibrium has been found.

Table 1. Comparing the STD of the results from different run.

Year	Std (%)
2013	3.46 × 10${}^{-12}$
2014	8.39 × 10${}^{-12}$
2015	3.82 × 10${}^{-12}$
2016	5.71 × 10${}^{-12}$
2017	4.98 × 10${}^{-12}$
2018	9.39 × 10${}^{-12}$
2019	3.04 × 10${}^{-12}$
2020	3.34 × 10${}^{-12}$

shows the results standard deviation (STD) as a percentage of the average from the game equilibrium of the experiment from the several runs, demonstrating that a stable Nash equilibrium was found for the problem. Notice that this table shows the standard deviation of the total financial result achieved at the equilibrium.

Figure 8 shows the average profile obtained for all the runs and compares them with the verified profile.

Once we obtained the best profile considering the non-cooperative game modeled and the spot price projections for the next year, we compared those results with the ones obtained with the actual verified seasonalization and at the verified spot price.

The financial results for the mechanism in case the game strategy was chosen are shown in

Table 2. Comparing the results from the verified profile versus the game proposed profile strategies.

Year	Verified Profile (R$ MM)	Game Proposed Profile (R$ MM)	Improvement (+)/Reduction (−) (R$ MM)
2013	9336	9552	+215
2014	663	1942	+1278
2015	−4412	−4418	−6
2016	−3189	−3609	−420
2017	−23,099	−21,914	+1186
2018	−18,439	−18,335	+104
2019	−10,128	−7319	+2809
2020	−11,268	−11,256	+67
TOTAL Gain:			+ 5233

, which presents the short-term settlement financial value for each case, showing the estimated short-term clearing that the entire mechanism would incur if one of the games strategy proposed was chosen.

Likewise, Table 2 shows the accumulated financial gain that the mechanism would attain if the agents could do a better evaluation of the results of different strategies and act by using this information, as if the mechanism was seasonalized using the equilibrium approach.

The total gain represents the sum of the short-term exposition of the seasonalization proposed by the Nash equilibrium over the years minus the sum of the realized seasonalization, and it demonstrates that the Nash equilibrium found, was able to improve the short-term exposition of the mechanism as a whole.

In addition, from this table, we may also infer that—despite the fact that the mechanism is a hedge instrument—the mechanism itself was not able to protect the hydro-generators from the severe dry period that has been taking place in Brazil over the last decade. That is the reason why the mechanism’s financial results are negative over the years from 2015 to 2020.

7. Discussion and Conclusions

Despite the simplifications made to reduce the size of the problem (there are more than 100 agents that may choose to seasonalize their PG), the price zone simplification is enough to put together the main characteristics that may have an effect on how each player would allocate their own PG—system hydro-generation and system PLD—and maintain the empirical numerical convergence guarantee.

So, the proposed game equilibrium model was capable of evaluating how the interaction among those four big agents could impact the financial result of the entire ERM. Although it did not take into account the particularity of each player, it was able to show how the interaction between those simplified agents would impact the financial result of the mechanism as a whole.

In order to solve the Nash equilibrium problem, we proposed a combined strategy that was able to sort the seasonalization game out. By using this tool, we were able to show different alternatives for the agents to allocate their energy simulating a rational non-cooperative game.

Despite the fact that there is always uncertainty in the PLD projections made at the end of each year, using the average forecast proved to be a good guide for the ERM agents to use as a reference for making their seasonalization decisions, since we simulated a non-cooperative game and obtained the Nash equilibrium for each year considering this kind of information.

In most of the years, the outcomes of the strategy proposed by the non-cooperative model allowed financial gains for the ERM as a whole. However, the solution the model proposed for year 2016 was worse than the strategy proposed by the agents. This result suggests that the use of the average forecast of the PLD is not enough to guarantee good results all the time and that a natural continuation of the present work would be to use more scenarios, and not only the average to assess the financial results. As future research, a model that considers the entire set of the projections available may be used to create a bi-objective optimization problem, by using a risk metric regarding the outcome of the agents. At this point, more agents could be added to the problem modeling their risk aversion preferences.

Finally, seasonalization is a process in which the decision of one agent has a direct impact on the result of the whole mechanism. This work proposed a tool to help agents evaluate what could be the best choice for the entire mechanism, considering it as a non-cooperative game. Based on that information, it is possible for each agent to use the profile achieved with the Nash equilibrium model as an input information for its own strategy. Moreover, this tool may be used by the policy makers to better evaluate the impact of the of the agents’ decisions in the entire ERM, improving the current rules in order to bring more competitiveness into the Brazilian energy market and, as a result, bring about cost reduction for consumers.