1. Introduction
The structure of electricity markets around the world has been profoundly transformed by the push towards liberalization in the late 90s and, more recently, by the massive arrival of renewable energy production. Distribution has been separated from production, and, where, in the past, a single producer could own the entire generation capacity of a given country or region, now a patchwork of small, often renewable, generators competes with a big historical producer.
The aim of this paper is to develop an equilibrium model for intraday electricity markets where a big producer with a significant market share competes with a large number of small renewable producers. The large producer and the small producers both use the intraday markets in order to compensate their production and demand forecast errors, creating feedback effects on the market price. The large producer can act strategically, anticipating the impact of its decisions on the market prices and, thus, on the behavior of the small agents. The small agents are not strategic, and each one has a negligible effect on the market; however, the behavior of all small agents taken together has a significant market impact. The large player has the first-mover advantage, but it does not observe the forecast of the minor players. These, in turn, have the information advantage, since they observe the forecast of the major player as well as their own forecast. This leads to a stochastic leader-follower game, where players interact through the market price. We place ourselves in the linear-quadratic setting, exhibit the unique Nash equilibrium for this game in closed form in the framework of mean field games with a major player, and provide explicit formulas for the market price and the strategies of the agents. For a game with a finite number of players, we show how an -Nash equilibrium can be constructed from the mean field game solution.
This paper is a companion paper to
Féron et al. (
2020), where a similar model is developed for the case of identical agents with symmetric interactions, and we refer the readers to that paper for a detailed review of literature on the stochastic and econometric modeling of intraday electricity markets. Here, we simply mention that a similar linear-quadratic setting with linear market impact has been used in order to determine optimal strategies for a single energy producer by
Aïd et al. (
2016) and
Tan and Tankov (
2018), while
Bouchard et al. (
2018) found an analytic expression for the equilibrium price in a linear-quadratic model of the stock market with symmetric interactions and perfect information. We also mention the recent paper of
Aïd et al. (
2020), where an equilibrium in complete information setting for a finite number of agents is derived in the intraday electricity market. This paper is close in spirit to the complete information framework of
Féron et al. (
2020), but it allows for treating the case of heterogeneous agents in conditions of uncertain production with possible outages and uncertain demand. However, the complete information setting, where each agent observes all other agents’ forecasts, does not seem to be realistic in electricity markets. The incomplete information setting, where each agent only observes its own forecast and the aggregate forecast, may not be tractable for a finite number of agents. Nevertheless, in
Féron et al. (
2020), it has been shown that explicit solutions may be found in the mean field limit, where the number of agents is sent to infinity, and the influence of every single agent on the entire market becomes negligible. The use of mean field theory for stochastic control with partial information has also recently been proposed by
Bensoussan and Yam (
2019), in a formal fashion, in order to solve the associated Zakai equation.
The mean field games (MFG) are stochastic differential games with infinitely many players and symmetric interactions. The seminal papers of
Lasry and Lions (
2007) and
Huang et al. (
2006) characterized the Nash equilibrium in this framework through a coupled system of a Hamilton–Jacobi–Bellman (HJB) and a Fokker–Planck (FP) equation.
Carmona and Delarue (
2018) developed an alternative probabilistic approach that was inspired by the Pontryagin principle and related the mean field game solution to a McKean–Vlasov Forward Backward Stochastic Differential Equation (FBSDE). The asymptotic results that were obtained in mean field games can be used to construct approximate equilibria (
-Nash equilibria) for games with a finite number of players. Alternatively, the equilibria of
N-player games can be shown to converge to the corresponding weak mean field equilibria
Lacker (
2020).
While the original MFG setting involves symmetric agents,
Huang (
2010) introduced linear-quadratic mean field games with a major player.
Nourian and Caines (
2013) developed this approach in a general framework. In both papers, the mean field is exogenous to the actions of the major player. In contrast to these two papers,
Bensoussan et al. (
2016) and
Carmona and Zhu (
2016) considered the endogenous case, where the major player can influence the mean field. In
Bensoussan et al. (
2016), this leads to a leader-follower setting, also known as Stackelberg game. The authors derived a HJB equation and a FP equation in order to characterize the solution in the general case, while the linear quadratic setting was tackled with a stochastic maximum principle approach. More recently,
Lasry and Lions (
2018) introduced a master equation accounting for this kind of major player model.
Cardaliaguet et al. (
2020) showed that the two previous approaches (
Lasry and Lions 2018;
Carmona and Zhu 2016) lead to the same Nash equilibria.
Financial markets and energy systems with many small interacting agents are a natural domain of applications of MFG.
Casgrain and Jaimungal (
2020) applied the MFG theory to optimal trade execution with price impact and terminal inventory liquidation condition,
Fujii and Takahashi (
2020), used this theory to find an equilibrium price under market clearing conditions. In
Casgrain and Jaimungal (
2020) and
Fujii and Takahashi (
2020), the authors used the extended mean field setting in order to deal with heterogeneous sub-populations of agents and incomplete information for
Casgrain and Jaimungal (
2020).
Alasseur et al. (
2020) developed a model for the optimal management of energy storage and distribution in a smart grid system through an extended MFG.
Shrivats et al. (
2020) recently applied the theoretical setting that was developed in
Casgrain and Jaimungal (
2020) to the case of trading in solar renewable energy certificate markets. Financial markets with a major player, leader–follower interactions, and terminal inventory constraint were recently analyzed in (
Evangelista and Thamsten 2020;
Fu and Horst 2020). In
Fu and Horst (
2020), the authors consider a Brownian filtration, impose a zero terminal inventory constraint and characterize the equilibrium in terms of a McKean-Vlasov FBSDE. In
Evangelista and Thamsten (
2020), the authors study a market with a finite number of small players and a major player with first-mover advantage and information asymmetry, and characterize the solution in terms of a McKean–Vlasov FBSDE in a more general setting than that of
Fu and Horst (
2020).
Among the cited papers, our methods and findings are the closest in spirit to (
Bensoussan et al. 2016;
Evangelista and Thamsten 2020;
Fu and Horst 2020). The main novelty of our paper is the application of the linear quadratic MFG with a major player to the analysis of electricity markets in the presence of renewable production; however, we also make a number of contributions to the mathematical theory. When compared to the article
Bensoussan et al. (
2016), which, of course, solves a more general problem, without focusing on a specific application, our paper allows a much more general dynamics for the driving processes (general semimartingales) and does not require an a priori bound on the strategies in order to prove the existence of the Nash equilibrium in the presence of a major player. Unlike the articles (
Evangelista and Thamsten 2020;
Fu and Horst 2020), which also study leader-follower games in financial markets, we consider a stochastic terminal constraint, characterize the equilibrium in explicit form, and show how an
-Nash equilibrium for the finite-player game may be constructed from a mean field game solution.
The paper is organized, as follows. In
Section 2, we introduce the model and briefly recall the mean field game solution obtained in
Féron et al. (
2020) in the case of identical agents. In
Section 3, we present the main results of this paper in the setting allowing for the presence of a major player, whose influence on the market is not negligible in the MFG limit. In
Section 4, we show how the limiting MFG solution may be used in order to construct an approximate Nash equilibrium in a Stackelberg game with one major player and
N minor players. Finally, in
Section 5, equilibrium price trajectories and the effect of market parameters on the price characteristics are illustrated with the simulated data.
2. Preliminaries
In this paper, we place ourselves in the intraday market for a given delivery hour starting at time T, where time 0 corresponds to the opening time of the market (in EPEX Intraday this happens at 3 p.m. on the previous day). In reality, trading stops a few minutes before delivery time (e.g., 5 min. for Germany). However, for the sake of simplicity, we assume that market participants can trade during the entire period . In the market, there are agents (producers or consumers) that are assumed to have taken a position in the day-ahead market and use the intraday market in order to manage the volume risk that is associated to the imperfect demand/production forecast. These forecasts represent the best estimate of the additional demand as compared to the position taken by the agent in the spot market: to avoid imbalance penalties, the intraday position of the agent at the delivery date must, therefore, be equal to the realized demand, or, in other words, the last observed value of the demand forecast.
We consider the case of a Stackelberg game, where an agent, called a “major agent”, faces a large number of smaller agents, called “minor agents”. We directly place ourselves in the setting of mean field games with a major player, that is, we assume that the number of small agents in the market is infinite, and the influence of each small agent on the market is negligible. Therefore, the aggregate impact of the minor agents on the market is modelled through a mean field.
Each agent observes the common national demand forecast and the demand forecast of the major player. In addition, the small agents also observe their individual demand forecasts, which are not observed by the other agents. Thus, the common filtration of the market contains information about the forecast of the major player and the common part of the forecasts of the minor players, but the small agents benefit from a private information advantage when compared to the major player.
The demand forecast process and position of the generic minor agent are given, respectively, by and , while the forecast process and position of the major agent are given, respectively, by and . Note that the position and forecast of the minor and major agents are not expressed in the same units. Indeed, in the mean field game limit considered in this paper, we assume that the market is very large, so that the position of every minor agent as compared to the market size is negligible, but the major agent takes up a nonzero share of the market, so that and denote the position and forecast of the major agent normalized by the market size.
We denote, by , the filtration that contains all information available to the generic minor agent and by the filtration, which contains all information available to the major agent. This filtration contains the information about the fundamental price, the information about the demand forecast of the major agent, and potentially some information regarding the demand forecast of the generic minor agent (the common noise), but, in general, not the full individual demand forecast of the generic minor agent.
Throughout the paper and for any
-adapted process
, we will denote
, where:
. In view of the convergence results of Proposition 9 and Proposition 10 in
Féron et al. (
2020), the (normalized) aggregate position of all minor agents is given by the expectation of
with respect to the common noise:
.
We assume that the market price
is given by the fundamental price
plus a weighted combination of the aggregate position of the minor agents and the position of the major agent:
where
a and
are positive weights, which reflect the size of the major agent relative to the combined size of all minor agents and overall strength of the market impact. Thus, the impact of each minor agent on the entire market is negligible, but the aggregate position of all minor agents and the position of the major agent both have a nonzero impact.
We say that the strategy of the generic minor agent
is admissible if it is
-adapted and square integrable. Similarly, the strategy of the major agent
is admissible if it is
-adapted and square integrable. The instantaneous cost of trading for the major agent and the generic minor agent are defined, respectively, by:
In both instantaneous costs, the first term represents the actual cost of buying the electricity and the second term represents the cost of trading, where and are continuous strictly positive functions on [0, T], reflecting the variation of market liquidity at the approach of the delivery date.
The objective function of the minor agent has the following form:
while the objective function of the major agent writes,
Note that this formulation implies (as is the case in real markets) that the major agent pays a much lower trading cost per unit traded and a much lower imbalance penalty than the minor agents. Indeed, if the major agent paid the same quadratic cost/penalty as the minor agents, since the position of the major agent is very large, the quadratic trading cost/penalty would grow much faster than the linear part (the middle term in the formula), and the limiting formula would degenerate, in the sense that the trading strategy would be independent from the price. In order to obtain a nondegenerate expression in terms of the normalized trading strategy of the major agent, we must, therefore, assume that the actual trading cost and penalty are also renormalized. The quantities
and
are, thus, different from
and
, since they are of different nature:
and
apply to the actual strategy of the generic agent, while
and
apply to the normalized strategy of the major agent. The different nature of trading costs for minor and major agents is confirmed by other authors (
Donier et al. 2015): while the minor agents post their orders immediately in the order book, the major agent splits its orders into many small chunks in order to minimize the trading costs.
To close this introductory section, we briefly recall one of the main results (Theorem 7) from
Féron et al. (
2020), which characterizes the mean field equilibrium in the setting of identical agents; in other words, we assume that
until the end of this section.
Definition 1 (mean field equilibrium).
An admissible strategy is a mean field equilibrium in the setting of identical agents if it maximizes the functional (2) with and satisfies . We make the following assumption.
Assumption 1. The process S is square integrable and adapted to the filtration .
The process X is a square integrable martingale with respect to the filtration .
The process that is defined by for is a square integrable martingale with respect to the filtration .
Note that, if X is an -martingale, then is by construction an -martingale, but it may not necessarily be a martingale in the larger filtration .
The following theorem characterizes the mean field equilibrium in the identical agent setting. In the theorem, we decompose the individual demand forecast, as follows:
, where
, and we use the following shorthand notation:
Theorem 1. Under Assumption 1, the unique mean field equilibrium in the setting of identical agents is given byThe equilibrium price has the following form: 3. A Game of a Major and Minor Agents
In this section, we proceed to characterize the Nash equilibrium in the Stackelberg mean field game with a major player. Because a single minor agent has an infinitesimal impact on the market and cannot influence the mean field or the strategy of the major agent, the problem of the generic minor agent is to maximize for fixed and . On the other hand, by modifying her strategy , the major agent may influence the strategies of the minor agents and, thus, also the mean field . This leads to the following definition of mean field equilibrium. As in the preceding section, the “consistency condition” in this definition simply translates the fact that the aggregate position of all minor agents is given by the expectation of the representative agent strategy with respect to the common noise.
Definition 2 (Stackelberg mean field equilibrium). We call the triple , , Stackelberg mean field equilibrium for the game with a major and minor players if the following holds:
- i.
and are admissible strategies for, respectively, the representative minor and the major players, the consistency condition is satisfied for all and for any other admissible strategy for the representative minor player ϕ, - ii.
For any other triple satisfying condition i.,
Assumption 2. In addition to Assumption 1, we also assume that the process is a square integrable martingale with respect to the filtration .
We start with the characterization of the optimal strategy for the minor agent.
Proposition 1 (Minor representative agent).
Let and be fixed. The minor agent strategy ϕ maximizes (2) over the set of admissible strategies if and only if:where Y is a -martingale that satisfies . Proof. The proof follows from the first step of the proof of Theorem 1 (see Theorem 7 in
Féron et al. (
2020)), taking
as fundamental price instead of
S. □
The problem of the major agent is more complex, since the minor agents observe the major agent’s actions and modify their strategies accordingly, which means that the mean field depends on the major agent’s strategy , and the problem of the major agent effectively becomes a stochastic control problem. We start with a reformulation of the definition of Stackelberg equilibrium in terms of and only.
Lemma 1. Let be -adapted square integrable processes. There exists , such that is a Stackelberg mean field equilibrium if and only if the couple satisfies the following conditions:
- i.
For every -adapted square integrable process ν, - ii.
For every other couple satisfying the condition i, the inequality (7) holds true.
Proof. First assume that
is a Stackelberg mean field equilibrium. Subsequently, for every
-adapted square integrable process
,
Developing the functionals we get,
and, since
is arbitrary, we see that this is equivalent to
Taking conditional expectations and using Fubini’s theorem, we then obtain condition i. of the lemma.
Assume now that conditions i. and ii. of the lemma hold true, and let
be given by Proposition 1 applied to the couple
. Define
. It remains to show that
. Let
be an
-martingale satisfying
. By integration by parts, condition i. of the lemma is equivalent to
and since
is arbitrary,
for all
t. On the other hand, by Proposition 1, while taking the expectation with respect to
, we get that there exists a
-martingale
with
, and such that
Substracting this expression from the previous one, we obtain
Thus,
and, therefore, using the terminal condition and the martingale property,
The unique solution of this linear equation is for all t, and, therefore, for all t. □
The following proposition provides a martingale characterization of the Stackelberg mean field equilibrium.
Proposition 2. Let be -adapted square integrable processes. There exists , such that is a Stackelberg mean field equilibrium if and only ifwhere is an -martingale and N is an absolutely continuous -adapted process, and there exists an -martingale M, and an -martingale , such that the following system of equations is satisfied: Proof. The optimization problem of the major agent consists in maximizing the objective function (
3) under the constraint of Lemma 1, part i. Following the methodology of
Bensoussan et al. (
2016), let us introduce the Lagrangian for this constrained optimization problem, which writes:
where
is a square integrable
-adapted process. We claim that
is the solution of the problem (
3) if and only if there exist
and
, such that
maximizes the Lagrangian
, and
satisfies the constraint of Lemma 1. Indeed, let
be such a triple and
be another pair of strategies satisfying the constraint of Lemma 1. Subsequently,
and, since both
and
satisfy the constraint of Lemma 1, this implies that inequality (
7) holds true.
We now turn to the problem of maximizing the Lagrangian. Let
. The first order condition for
writes: there exists a martingale
, such that
The first order condition for
writes: there exists a martingale
M, such that
Finally, the last condition is given by the constraint that
is optimal for the generic minor agent. Hence, conditioning (
8) by the common noise, there exists a martingale
, such that
□
From Proposition 2, it follows that the existence and uniqueness of the Stackelberg equilibrium reduces to the existence and uniqueness of the solution of the linear system of coupled BSDEs Equation (
10).
The following theorem provides an explicit characterization of the equilibrium in the Stackelberg setting.
Theorem 2 (Explicit solution).
Let . The following differential equation characterizes the unique equilibrium of the mean field game with a major agent:where N is a -adapted process with and , M, and are -martingales that satisfy:andDenoting, by , the fundamental matrix solution of the equation , the solution is given in integral form by the following expression:where,.
Remark 1. These results can be generalized to the setting of several major agents, interacting with the mean-field of the minor agents, provided that each major agent observes the individual forecasts of the other agents, but not those of the minor agents. In this case, the constrained optimization problem of one major agent becomes a constrained game between several major agents. Because the setting remains linear-quadratic, one will still be able to obtain an explicit solution, at the price of more tedious computations.
Remark 2. If for some constant c, the fundamental matrix solution is explicitly given by Proof. From Equation (
8) in Proposition 1 and (
10) in Proposition 2, we immediately deduce the expression of the characterizing differential equation of the equilibrium (
11).
Let
be the fundamental matrix solution of the equation
, which is, for every
, the solution with initial condition
is given by
. By a variation of constants, we have that the solution of (
11) is given by:
Letting:
we obtain the simplified equation:
and, finally, using (
12) and the martingale property, the martingale components satisfy:
From this, we deduce, on the one hand,
and, on the other hand,
so that, finally:
□
Let us make some comments regarding how minor and major player strategies change when the parameters of the model vary. First, when the major player has no price impact,
, we recover the homogeneous mean field setting optimal strategy for the minor player from (
10) and (
12):
where
satisfies the equation:
Second, we explore the limiting behavior of the optimal strategies for the major agent and mean field in various limiting cases. In this corollary, we use the notation of Theorem 2.
Corollary 1. - i.
Assume that the fundamental price process S is a martingale. Subsequently, the equilibrium mean field position of minor agents and the position of the major agent satisfy - ii.
In the limit of infinite terminal penalty (when ), the equilibrium mean field position of minor agents and the position of the major agent satisfy, almost surely for all , where When the fundamental price process S is a martingale, in the limit of infinite terminal penalty, the strategies do not depend on the fundamental price and we have, - iii.
In the absence of terminal penalties (when ), the equilibrium mean field position of minor agents and the position of the major agent satisfy,
Proof. The first part is a simplification of the proof of Theorem 2. Using the expressions of
and
in Theorem 2 and the martingale property of
, we can rewrite:
Substituting these expressions in the Equation (
13), we obtain the result.
For the second part, we can rewrite:
and when
,
and
. The third part follows by direct substitution of
into the general formula. □
Interestingly, when the players do not have a terminal penalty (
), the equilibrium positions of the agents in Equation (
14) still contain forward looking terms, which were absent in the case of the mean field game with identical players (see Equation (
5) with
). The presence of these terms is due to the strategic interaction of the major player with the mean field of small agents.
In the limit of zero trading costs, the gain of the major player remains bounded in expectation; however, contrary to the case of identical players, the optimal strategy of the major agent cannot be uniquely determined from the optimization problem. Indeed, while assuming that the trading cost for minor agents is zero, the equilibrium price (computed from Equation (
12) in
Féron et al. (
2020) with
) is given by
Substituting this expression into the optimization problem for the major player, we need to minimize the following functional:
where the equality follows, in particular, from the martingale property of
. Because the expression to be minimized only depends on the terminal value
of the major agent’s position, any strategy with the optimal terminal value will satisfy the condition of optimality: the Stackelberg equilibrium will not be unique in this case.
To finish this section, we provide the explicit form of the strategy of the minor agents.
Corollary 2 (Minor agent strategy).
Under Assumption 2, the optimal generic minor agent position is given by:where is the optimal aggregate position of the minor agents, as given by Theorem 2. Proof. Let
,
and
. Subsequently, from the explicit form of
Y and
in Proposition 1, it follows that
is an
-martingale and it satisfies
Subsequently,
and by the martingale property,
Solving this linear equation for
and then substituting into (
15), we obtain the result. □
4. Approximate Nash Equilibrium in the N-Player Stackelberg Game
In this section, we derive the -Nash approximation for the Stackelberg game. In the present leader-follower setting, we allow the minor agents to change their strategies when the major agent deviates from the optimal one.
Because we would like to study the rate of convergence as , we assume that there is a major player and an infinity of minor players replacing the generic agent. Their demand forecasts are, respectively, given by , , and . The private demand forecasts of all agents are defined on the same probability space. Therefore, we impose the following assumption.
Assumption 3. The process S is square integrable and adapted to the filtration .
The demand forecast of the major agent is a square integrable -martingale.
The processes are square integrable -martingales.
There exists a square intergrable -martingale , such that for all , and all , almost surely, .
The processes that are defined by for , are orthogonal square integrable -martingales, such that the expectation does not depend on i.
The strategy
of agent
is said to be admissible if it is
-adapted and square integrable; the strategy
of the major agent is admissible if it is
-adapted and square integrable. For a fixed
, we denote:
where
is the average position of the minor agents. Additionally, we define in the
N-player game, the objective functions for the major agent:
and for the minor agents
:
as well as the objective function for the minor agents
, in the mean field setting:
We next provide a definition of the
-Nash equilibrium in the present Stackelberg setting. As mentioned above, the deviations of the major and minor agents must be treated differently: when the major agent deviates, we allow the minor agents to adjust their strategies to respond optimally to the new strategy of the major agent. We say that the minor agent strategies
are an optimal response to the major agent strategy
if, for every
and for every admissible minor agent strategy
,
Definition 3 (Stackelberg -Nash equilibrium). We say that is an ϵ-Nash equilibrium for the N-player game if these strategies are admissible and the following holds.
- i.
Deviation of a minor player:for any other admissible strategy for the minor player i, , - ii.
Deviation of the major player:for any other set of admissible strategies , such that are optimal responses of minor players to the major player strategy , we have,
Our definition of
-Nash equilibrium is different from the one presented in
Carmona and Zhu (
2016): while the latter paper assumes that the major player deviates from her strategy unilaterally (see Definition 4.2 in the cited paper), we allow the minor players to respond to the deviation of the major player, in agreement with the leader-follower nature of the game. In addition, in
Carmona and Zhu (
2016), an a priori bound on the
-norm of the new strategy of the major agent is required to establish Theorem 4.1 in the cited paper, whereas no such bound is needed in our setting.
Proposition 3. Assume that the strategies of the N minor agents are given bywhere is the third component of the mean field equilibrium defined in Theorem 2. Assume that the strategy of the major agent is also given by Theorem 2. Let Assumption 3 hold true. Then there exists a constant , which does not depend on N, such that these strategies form an ε-Nash equilibrium of the N-player game with . Remark 3. The ε-Nash equilibrium that is described in Proposition 3 approximates the N-player equilibrium in the complete information setting (where every player observes the others’ actions), but its implementation for each agent only requires the knowledge of the common information as well as the agent’s individual forecast.
Proof. We need to show the conditions i. and ii. of Definition 3. Condition i. is shown in the same way as in the case of homogeneous players (see proof of Proposition 2 in
Féron et al. 2020). Therefore, we focus on condition ii. Assume that all of the agents change their strategies to new ones
, such that
are optimal responses to
. Let
be the optimal "mean field" response to the major agent strategy
.
Step 1.
We first suppose that there exists a finite constant , independent of N, such that .
By Proposition 1 in
Féron et al. (
2020), for some constants
c and
C, which do not depend on
N, and may change from line-to-line,
and by our assumption,
Thus, while using Cauchy–Schwartz inequality,
Step 2.
Let there exist a sufficiently large constant
A, independent of
N, such that
Letting
, and
, we have, by definition,
On the other hand, because
for
are optimal responses to
, we get that
where
is defined by
. Summing up the above inequality over
, dividing by
N and using Jensen’s inequality, we obtain
Multiplying this inequality by
, adding it to the first one, and using integration by parts, we finally obtain
Thus, if
for
A sufficiently large (but not depending on
N), then, from the above estimate, it follows that
as well. □