Mean-Field Type Games between Two Players Driven by Backward Stochastic Differential Equations

Abstract

In this paper, mean-field type games between two players with backward stochastic dynamics are defined and studied. They make up a class of non-zero-sum, non-cooperating, differential games where the players’ state dynamics solve backward stochastic differential equations (BSDE) that depend on the marginal distributions of player states. Players try to minimize their individual cost functionals, also depending on the marginal state distributions. Under some regularity conditions, we derive necessary and sufficient conditions for existence of Nash equilibria. Player behavior is illustrated by numerical examples, and is compared to a centrally planned solution where the social cost, the sum of player costs, is minimized. The inefficiency of a Nash equilibrium, compared to socially optimal behavior, is quantified by the so-called price of anarchy. Numerical simulations of the price of anarchy indicate how the improvement in social cost achievable by a central planner depends on problem parameters.

1. Introduction

Mean-field type games (MFTG) is a class of games in which payoffs and dynamics depend not only on the state and control profiles of the players, but also on the distribution of the state-control processes. MFTGs has by now a plethora of applications in the engineering sciences, see [1] and the references therein. This paper studies MFTGs between two players, with state-distribution dependent cost functionals $J^i:{\mathcal{U}}^i\to \mathbb{R}$, $i=1,2$, and mean-field BSDE state dynamics. The Nash solution $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$ is dictated by the pair of inequalities

$$\begin{array}{c}\hfill J^1({\widehat{u}}_{·}^1;{\widehat{u}}_{·}^2)\le J^1(u_{·}^1;{\widehat{u}}_{·}^2),\forall u_{·}^1\in {\mathcal{U}}^1,\\ \hfill J^2({\widehat{u}}_{·}^2;{\widehat{u}}_{·}^1)\le J^2(u_{·}^2;{\widehat{u}}_{·}^1),\forall u_{·}^2\in {\mathcal{U}}^2.\end{array}$$

(1)

Following the path laid-out in [2], we establish a Pontryagin type maximum principle, yielding necessary and sufficient conditions for any pair of controls satisfying (1). Behavior in the equilibrium (1) is compared to the socially optimal solution, that minimizes the social cost $J:=J^1+J^2$

$$J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\le J(u_{·}^1,u_{·}^2),\forall u_{·}^1\in {\mathcal{U}}^1,u_{·}^2\in {\mathcal{U}}^2.$$

(2)

1.1. Related Work

Pontryagin’s maximum principle is the tool, alongside dynamic programming, to characterize optimal controls in both deterministic and stochastic settings. It can treat not only standard stochastic systems, but generalizes to optimal stopping, singular controls, risk-sensitive controls and partially observed models. Pontryagin’s maximum principle yields necessary conditions that must be satisfied by any optimal solution. The necessary conditions become sufficient under additional convexity conditions. Early results showed that an optimal control along with the corresponding optimal state trajectory must solve the so-called Hamiltonian system, which is a two-point (forward-backward) boundary value problem, together with a maximum condition of the so-called Hamiltonian function. A very useful aspect of this result is that minimization of the cost functional (over a set of control functions) may reduce to pointwise maximization of the Hamiltonian, at each point in time (over the set of control values). Pontryagin’s technique for deterministic systems and stochastic systems with uncontrolled diffusion can be summarized as follows: assume that there exists an optimal control, make a spike-variation of it and then consider the first order term of the Taylor expansion with respect to the perturbation. This leads to variational inequality and the result follows from duality. If the diffusion is controlled, second order terms in the Taylor expansion has to be considered. In this case, one ends up with two forward-backward SDEs and a more involved maximum condition for the Hamiltonian. See [3] for a detailed account. For stochastic systems, the backward equation is fundamentally different from the forward equation, whenever one is looking for adapted solutions. An adapted solution to a BSDE is a pair of adapted stochastic processes $(Y_{·},Z_{·})$, where $Z_{·}$ corrects “non-adaptiveness” caused by the terminal condition of $Y_{·}$. As pointed out by [4], the first component $Y_{·}$ corresponds to the mean evolution of the dynamics, and $Z_{·}$ to the risk between current time and terminal time. Linear BSDEs extends to non-linear BSDEs [5], with applications not only within stochastic optimal control but also in stochastic analysis [6] and finance [7,8], and to forward-backward SDEs (FBSDE). BSDEs with distribution-dependent coefficients, mean-field BSDEs, were derived as limits of particle systems in [9]. Existence and uniqueness results for mean-field BSDEs, as well as a comparison theorem, are provided in [10].

In stochastic differential games, both zero-sum and nonzero-sum, Pontryagin’s stochastic maximum principle (SMP) and dynamic programming are the main tools for obtaining conditions for an equilibrium. These tools were essentially inherited from the theory of stochastic optimal control. As in the optimal control setting, the latter deals with solving systems of second-order parabolic partial differential equations, while the former is related to analyzing FBSDEs where, in the case of initial state constraints, the BSDE corresponds to the adjoint process. For a recent example of the use of SMP in stochastic differential game theory, see [11].

The theory of mean-field type control (MFTC), initiated in [2], treats stochastic control problems with coefficients dependent on the marginal state-distribution. This theory is by now well developed for forward stochastic dynamics, i.e., with initial conditions on state [12,13,14,15]. With SMPs for MFTC problems at hand, MFTG theory can inherit these techniques like stochastic differential game theory does in the mean-field free case. See [16] for a review of solution approaches to MFTGs. A MFTC problem can be interpreted as a large population limit of a cooperative game, where the players share a joint goal to optimize some objective [17]. A close relative to MFTC is mean field games (MFG). MFGs is class of non-cooperative stochastic differential games where a large number of indistinguishable (anonymous) players interact weakly through a mean-field coupling term, initiated by [18,19] independently, and followed up by, among many others [20,21,22]. Weak player-to-player interaction through a mean-field coupling restricts the influence one player has on any other player to be inversely proportional to the number of players, hence the level of influence of any specific player is very small. The coupling of player state dynamics leads to conflicting objectives, and an approximation of mass behavior provides an approximate limit solution (equilibrium) to the MFG. In contrast to the MFG, players in a MFTG can be influential, and distinguishable (non-anonymous). That is, state dynamics and/or cost need not be of the same form over the whole player population, and a single player can have a major influence on other players’ dynamics and/or cost.

Already in [23], an SMP in local form was derived for a controlled non-linear BSDE. By first finding a global estimate for the variation of the second component of the BSDE solution, an SMP in global form was derived in [24]. A reinterpretation of BSDEs as forward stochastic optimal control problems [4] opened up for a new solution approach in the field of control of BSDEs. Inspired by the reinterpretation, optimal control of LQ BSDEs was solved in [25] by constructing a sequence of forward control problems with an additional state constraint, whose limit solution is the solution to the original LQ BSDE control problem. This approach was later used by [26] to solve a general FBSDE control problem, where the authors overcome the difficulty of controlling the diffusion in the forward process. Instead of writing down a second-order adjoint equation for the full system, the technique of [25] is used. Previous to that, [27] studied optimal control problem for general coupled forward–backward stochastic differential equations (FBSDEs) with controlled diffusions. A maximum principle of Pontryagin’s type for the optimal control is derived, by means of spike variation techniques.

Optimal control of mean-field BSDEs has recently gained attention. In [28] the mean-field LQ BSDE control problem with deterministic coefficients is studied. Assuming the control space is linear, linear perturbation is used to derive a stationarity condition which together with a mean-field FBSDE system characterizes the optimal control. Existence of optimal controls is also proven under convexity assumptions. Other recent work on the control of BSDEs includes [29,30], both using the FBSDE approach of [27].

1.2. Potential Applications of MFTG with Mean-Field BSDE Dynamics

In [31], a model is proposed for pedestrians groups moving towards targets they are forced to reach, such as deliveries and emergency personnel. The strict terminal condition leads to the formulation of a dynamic model for crowd motion where the state dynamics is a mean-field BSDE. Mean-field effects appear in pedestrian crowd models as approximations of aggregate human interaction, so the game would in fact be a MFTG [32]. A game between such groups is of interest since it can be a tool for decentralized decision making under conflicting interests. Other areas of application include strategies for financial investments, where often future conditions are specified [8,33] and lead to dynamic models including BSDEs. The already mentioned study [1] presents a lengthy list of applications of forward MFTGs in engineering sciences.

1.3. Paper Contribution and Outline

In this paper, control of mean-field BSDEs is extended to games between players whose state dynamics are mean-field BSDEs. Such games are in fact MFTGs, since the distribution of each player is effected by both players’ choice of strategy. Our MFTG could be viewed as a game between mean-field FBSDEs, where the backward equation is the state equation, and the forward equation is pure noise. A Pontryagin’s type SMP is derived, resulting in a verification theorem and conditions for existence of a Nash equilibrium. This solution approach is similar to that of [23,24,28]. The use of spike-perturbation requires minimal assumptions on the set of admissible controls, and differentiating measure-valued functions makes it possible to go beyond linear-quadratic mean-field cost and dynamics. The state BSDE is not converted to a forward optimization problem in the spirit of [25]. As a consequence, the adjoint equation in our SMP is a forward SDE. For the sake of comparison, optimality conditions for the cooperative situation are derived. In this setting, the players work together to optimize social cost, which is the the sum of player costs. The approach used is a straight-forward adaptation of the techniques used in control of SDEs of mean-field type; again, we do not need to take the route via some equivalent forward optimization problem to solve the backward MFTC problem. This cooperative game is a MFTC problem, and our result here is basically a special case of the FBSDE results in [27] or [26] mentioned above, although mean-field terms are present. Numerical simulations are done in the linear-quadratic case, which is explicitly solvable up to a system of ODEs. The examples pinpoint differences between player behavior in the game versus the centrally planned solution. The fraction between the social cost in the game equilibrium and the social cost optimum quantifies the game efficiency and was first studied in [34] for traffic coordination on networks under the name coordination ratio. This fraction was later renamed to the price of anarchy in [35]. We notice that paying a high price for using large control values, or deviating from a preferred initial position makes the problem stiffer, in the sense that the improvement by team optimality is decreasing, while paying a high price for mean-field related costs makes the problem less stiff.

The rest of this paper is organized as follows. The problem formulation is given in Section 2. Section 3 and Section 4 deal with necessary and sufficient conditions for any Nash equilibrium and social optimum; maximum principles for the MFTG and the MFTC are derived. An LQ problem is solved explicitly in Section 5, and numerical results are presented. The paper concludes with some remarks on possible extensions in Section 6, followed by an appendix containing proofs.

2. Problem Formulation

List of Symbols

• $T\in (0,\infty )$—the time horizon.
• $(\mathrm{\Omega },\mathcal{F},\mathbb{F},\mathbb{P})$—the underlying filtered probability space.
• $\mathcal{L}(X)$—the distribution of a random variable X under $\mathbb{P}$.
• $L_{{\mathcal{F}}_t}^2(\mathrm{\Omega };{\mathbb{R}}^d)$—the set of ${\mathbb{R}}^d$-valued ${\mathcal{F}}_t$-measurable random variables X such that $\mathbb{E}[|X{|}^2]<\infty$.
• $\mathcal{G}$—the progressive $\sigma$-algebra.
• $X_{·}$—a stochastic process ${\left\{X_t\right\}}_{t\ge 0}$.
• ${\mathbb{S}}^{2,k}$—the set of ${\mathbb{R}}^k$-valued, continuous $\mathcal{G}$-measurable processes $X_{·}$ such that $\mathbb{E}[{\displaystyle \underset{t\in [0,T]}{sup}}|X_t{|}^2]<\infty$.
• ${\mathbb{H}}^{2,k}$—the set of ${\mathbb{R}}^k$-valued $\mathcal{G}$-measurable processes $X_{·}$ such that $\mathbb{E}[{\int }_0^T|X_s{|}^{2}ds]<\infty$.
• ${\mathcal{U}}^i$— the set of admissible controls for player i.
• $\mathcal{P}(\mathcal{X})$—the set of probability measures on $\mathcal{X}$.
• ${\mathcal{P}}_2(\mathcal{X})$—the set of probability measures on $\mathcal{X}$ with finite second moment.
• ${\Theta }_t^i$—the t-marginal of the state-, law- and control-tuple of player i.
• ${\parallel Z\parallel }_F$—the trace (Frobenius) norm of the matrix Z.
• ${\partial }_{y^i}f(y^i)$—derivative of the ${\mathbb{R}}^d$-valued function f.
• ${\partial }_{{\mu }^i}f({\mu }^i)$—derivative of the ${\mathcal{P}}_2({\mathbb{R}}^d)$-valued function f, see Appendix A for details.

Let $T>0$ be a finite real number representing the time horizon of the game. Consider a filtered probability space $(\mathrm{\Omega },\mathcal{F},{\left\{{\mathcal{F}}_t\right\}}_{t\ge 0},\mathbb{P})$ on which two independent standard Brownian motions $W_{·}^1,W_{·}^2$ are defined, $d_1$- and $d_2$-dimensional respectively. Additionally, $y_T^1,y_T^2\in L_{{\mathcal{F}}_T}^2(\mathrm{\Omega };{\mathbb{R}}^d)$ and $\xi$, ${\mathcal{F}}_0$-measurable, are defined on the space. We assume that these five random objects are independent and that they generate the filtration $\mathbb{F}:={\left\{{\mathcal{F}}_t\right\}}_{t\ge 0}$. Notice that $\xi$ makes ${\mathcal{F}}_0$ non-trivial. Let $\mathcal{G}$ be the $\sigma$-algebra on $[0,T]\times \mathrm{\Omega }$ of ${\mathcal{F}}_t$-progressively measurable sets. For $k\ge 1$, let ${\mathbb{S}}^{2,k}$ be the set of ${\mathbb{R}}^k$-valued and continuous $\mathcal{G}$-measurable processes $X_{·}:=\{X_t:t\in [0,T]\}$ such that $\mathbb{E}[{sup}_{t\in [0,T]}|X_t{|}^2]<\infty$, and let ${\mathbb{H}}^{2,k}$ be the set of ${\mathbb{R}}^k$-valued $\mathcal{G}$-measurable processes $X_{·}$ such that $\mathbb{E}[{\int }_0^T|X_s{|}^{2}ds]<\infty$.

Let $(U^i,d_{U^i})$ be a separable metric space, $i=1,2$. Player i picks her control $u_{·}^i$ from the set

$${\mathcal{U}}^i:=\{u:[0,T]\times \mathrm{\Omega }\to U^i|u_{·}\mathrm{is}\mathbb{F}\mathrm{-adapted},\mathbb{E}[{\int }_0^T{d}_{U^i}{(u_s)}^{2}ds]<\infty \}.$$

(3)

The distribution of any random variable $\xi \in \mathcal{X}$ will be denoted by $\mathcal{L}(\xi )\in \mathcal{P}(\mathcal{X})$, and $-i$ will denote the index $\{1,2\}\backslash i$. Given a pair of controls $(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$, consider the system of controlled BSDEs

$$d{Y}_t^i=b^i(t,{\Theta }_t^i,{\Theta }_t^{-i},Z_t)dt+Z_t^{i,1}d{W}_t^1+Z_t^{i,2}d{W}_t^2,Y_T^i=y_T^i,i=1,2,$$

(4)

where ${\Theta }_t^i=(Y_t^i,\mathcal{L}(Y_t^i),u_t^i)$ and $Z_t=[Z_t^{1,1}{Z}_t^{1,2}{Z}_t^{2,1}{Z}_t^{2,2}]$. Furthermore,

$$b^i:\mathrm{\Omega }\times [0,T]\times S\times U^i\times S\times U^{-i}\times {\mathbb{R}}^{d\times (2d_1+2d_2)}\to {\mathbb{R}}^d,$$

(5)

where $S:={\mathbb{R}}^d\times \mathcal{P}({\mathbb{R}}^d)$ is equipped with the norm ${\parallel (y,\mu )\parallel }_S:=|y|+d_2(\mu )$, $d_2$ being the 2-Wasserstein metric on $\mathcal{P}({\mathbb{R}}^d)$. ${\mathbb{R}}^{d\times (2d_1+2d_2)}$ is equipped with the trace norm ${\parallel Z\parallel }_F=\mathrm{tr}{(Z{Z}^{\ast })}^{1/2}$. Note that if X is a square integrable random variable in ${\mathbb{R}}^d$, then $d_2(\mathcal{L}(X))<\infty$ and $\mathcal{L}(X)\in {\mathcal{P}}_2({\mathbb{R}}^d)$, the space of measures with finite $d_2$-norm.

Given $(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$, a pair of ${\mathbb{R}}^d\times {\mathbb{R}}^{d\times (d_1+d_2)}$-valued $\mathcal{G}$-measurable processes $(Y_{·}^i,[Z_{·}^{i,1}{Z}_{·}^{i,2}])$, $i=1,2$, is a solution to (4) if

$$Y_t^i=y_T^i-{\int }_t^T{b}^i(s,{\Theta }_s^i,{\Theta }_s^{-i},Z_s)ds-\sum _{j=1}^2{\int }_t^T{Z}_s^{i,j}d{W}_s^j,\forall t\in [0,T],a.s.,$$

(6)

and $(Y_{·}^i,[Z_{·}^{i,1}{Z}_{·}^{i,2}])\in {\mathbb{S}}^{2,d}\times {\mathbb{H}}^{2,d\times (d_1+d_2)}$.

Remark 1.

Any terminal condition $y_T^i\in L_{{\mathcal{F}}_T}^2(\mathrm{\Omega };{\mathbb{R}}^d)$ naturally induces a $\mathbb{F}$-martingale $Y_t^i:=\mathbb{E}[y_T^i|{\mathcal{F}}_t]$. The martingale representation theorem then gives existence of a unique process $[Z_{·}^{i,1},Z_{·}^{i,2}]\in {\mathbb{H}}^{2,d\times (d_1+d_2)}$ such that $Y_t^i=y_T^i+{\int }_t^T{Z}_s^{i,1}d{W}_s^1+{\int }_t^T{Z}_s^{i,2}d{W}_s^2$, i.e., $[Z_{·}^{i,1},Z_{·}^{i,2}]$ plays the role of the projection and without it, $Y_{·}^i$ would not be $\mathcal{G}$-measurable. Hence the noise $(W_{·}^1,W_{·}^2)$ generating the filtration is common to both players, and $[Z_{·}^{i,1},Z_{·}^{i,2}]$, $i=1,2$ is their respective reaction to it. Player i may actually be effected by all the noise in the filtration even if only some components of $(W_{·}^1,W_{·}^2)$ appear in $b^i$. An interpretation of $[Z_{·}^{i,1},Z_{·}^{i,2}]$ is that it is a second control of player i: first she plays $u_{·}^i$ to heed preferences on energy use, initial position etc., then she picks $[Z_{·}^{i,1},Z_{·}^{i,2}]$ so that her path to $y_T^i$ is the optimal prediction based on available information in the filtration at any given time. The component $b^i$ in (4) acts as a velocity in.

Existence and uniqueness of (4) is given by a slight variation of the results of [10], where the one-dimensional case is treated. For the d-dimensional mean-field free case, see [36].

Assumption 1.

The process $b^i(\omega ,·,0,\cdots ,0)$, $i=1,2$, belongs to ${\mathbb{H}}^{2,d}$ and for any $v^i=(y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z)\in S\times U^1\times S\times U^2\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$, $b^i(\omega ,·,v^i)$, $i=1,2$, is $\mathcal{G}$-measurable.

Assumption 2.

Given a pair of control values $(u^1,u^2)\in U^1\times U^2$, there exists a constant $L>0$ such that for all $t\in [0,T]$ and tuples $(y^1,{\mu }^1,y^2,{\mu }^2,z),({\overline{y}}^1,{\overline{\mu }}^1,{\overline{y}}^2,\overline{{\mu }^2},\overline{z})\in S\times S\times {\mathbb{R}}^{d(2d_1+2d_2)}$

$$\begin{array}{c}|b^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z)-b^i(t,{\overline{y}}^i,{\overline{\mu }}^i,u^i,{\overline{y}}^{-i},{\overline{\mu }}^{-i},u^{-i},\overline{z})|\hfill \\ \le L\left(\sum _{j=1}^2\parallel (y^j,{\mu }^j)-({\overline{y}}^j,{\overline{\mu }}^j){\parallel }_S+{\parallel z-\overline{z}\parallel }_F\right),\mathbb{P}-\mathrm{a}.\mathrm{s}.,i=1,2.\hfill \end{array}$$

(7)

Theorem 1.

Let Assumptions 1 and 2 hold. Then, for any terminal conditions $y_T^1,y_T^2\in L^2(\mathrm{\Omega },{\mathcal{F}}_T,\mathbb{P};{\mathbb{R}}^d)$ and $(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$, the system of mean-field BSDEs (4) has a unique solution $(Y_{·}^i,[Z_{·}^{i,1},Z_{·}^{i,2}])\in {\mathbb{S}}^{2,d}\times {\mathbb{H}}^{2,d\times (d_1+d_2)}$, $i=1,2$.

Next, we introduce the best reply of player i as follows:

$$J^i(u_{·}^i;u_{·}^{-i}):=\mathbb{E}[{\int }_0^T{f}^i(t,{\Theta }_t^i,{\Theta }_t^{-i})dt+h^i(Y_0^i,\mathcal{L}(Y_0^i),Y_0^{-i},\mathcal{L}(Y_0^{-i}))]$$

(8)

for given maps $f^i:[0,T]\times S\times U^i\times S\times U^{-i}\to \mathbb{R}$ and $h^i:\mathrm{\Omega }\times S\times S\to \mathbb{R}$.

Assumption 3.

For any pair of controls $(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$, $f^i(·,{\Theta }_{·}^i,{\Theta }_{·}^{-i})\in L_{\mathcal{F}}^1(0,T;\mathbb{R})$ and $h(Y_0^i,\mathcal{L}(Y_0^i),Y_0^{-i},\mathcal{L}(Y_0^{-i}))\in L_{{\mathcal{F}}_0}^1(\mathrm{\Omega };\mathbb{R})$.

The problems we consider next are

• The Mean-field Type Game (MFTG): find the Nash equilibrium controls of

  $$\left\{\begin{array}{cc}\underset{u_{·}^i\in {\mathcal{U}}^i}{inf}\hfill & J^i(u_{·}^i;u_{·}^{-i}),i=1,2,\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & d{Y}_t^i=b^i(t,{\Theta }_t^i,{\Theta }_t^{-i},Z_t)dt+Z^{i,1}d{W}_t^1+Z_t^{i,2}d{W}_t^2,Y_T^i=y_T^i.\hfill \end{array}\right.$$

  (9)
• The Mean-field Type Control Problem (MFTC): find the optimal control pair of

  $$\left\{\begin{array}{cc}\underset{(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2}{inf}\hfill & J(u_{·}^1,u_{·}^2):=J^1(u_{·}^1;u_{·}^2)+J^2(u_{·}^2;u_{·}^1),\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & d{Y}_t^i=b^i(t,{\Theta }_t^i,{\Theta }_t^{-i},Z_t)dt+Z^{i,1}d{W}_t^1+Z_t^{i,2}d{W}_t^2,Y_T^i=y_T^i,\hfill \\ & i=1,2.\hfill \end{array}\right.$$

  (10)

In the game each player assumes that the other player acts rationally, i.e., minimizes cost, and picks her control as the best response to that. This leads to a set of two inequalities, characterizing any control pair $(u_{·}^1,u_{·}^2)$ that constitute a Nash equilibrium. In this paper, each player is aware of the other player’s control set, best response function and state dynamics. Therefore, even though the decision process is decentralized, both players solve the same set of inequalities. When there is not a unique Nash equilibrium, there is an ambiguity around which equilibrium strategy to play if the players do not communicate. In the control problem, a central planner decides what strategies are played by both of the players. The central planner might just be the two players cooperating towards a common goal, or some superior decision maker. The goal is to find the control pair that minimizes the social cost J. This notion of a centrally planned/cooperative solution is related to the concept of team optimality in team problems [37]. In a team problem, the players share a common objective. A team-optimal solution is then the solution to the joint minimization of the common objective. In our case, the social cost J is a common objective in the MFTC. The Nash solution to the team problem is given by the control pair that satisfies the two inequalities

$$\begin{array}{c}J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\le J(u_{·}^1,{\widehat{u}}_{·}^2),\forall u_{·}^1\in {\mathcal{U}}^1,\hfill \\ J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\le J({\widehat{u}}_{·}^1,u_{·}^2),\forall u_{·}^2\in {\mathcal{U}}^2.\hfill \end{array}$$

(11)

In (11), each player is minimizing the social cost with respect to its marginal, under the assumption that the other player is minimizing its marginal. This is the so-called player-by-player optimality of a control pair in a team problem. Notice that if we set $J^1(u_{·}^1;u_{·}^2)=J^2(u_{·}^2;u_{·}^1)$ in (9), it becomes a team problem. The solution to the MFTG (9) will then be the player-by-player optimal solution to the minimization of the social cost.

Logically, we expect the optimal social cost to be lower than the social cost in a Nash equilibrium. The ratio between the worst case social cost in the game and the optimal social cost is called the price of anarchy, and we will highlight it in the numerical simulations in Section 5 where we also observe behavioral differences between MFTG and MFTC given identical data.

3. Problem 1: MFTG

This section is the derivation of necessary and sufficient equilibrium conditions of (9). Given the existence of such a pair of controls, we derive the conditions by the means of a Pontryagin type stochastic maximum principle.

Assume that $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$ is a Nash equilibrium for the MFTG, i.e., satisfies the following system of inequalities,

$$\left\{\begin{array}{c}{J}^1({\widehat{u}}_{·}^1;{\widehat{u}}_{·}^2)\le J^1(u_{·}^1;{\widehat{u}}_{·}^2),u_{·}^1\in {\mathcal{U}}^1,\hfill \\ J^2({\widehat{u}}_{·}^2;{\widehat{u}}_{·}^1)\le J^2(u_{·}^2;{\widehat{u}}_{·}^2),u_{·}^2\in {\mathcal{U}}^2.\hfill \end{array}\right.$$

(12)

Consider the first inequality, with ${\overline{u}}^{\epsilon ,1}$ chosen as a spike-perturbation of ${\widehat{u}}^1$. That is, for $u_{·}\in {\mathcal{U}}^1$,

$${\overline{u}}_t^{\epsilon ,1}:=\left\{\begin{array}{cc}{\widehat{u}}_t^1,\hfill & t\in [0,T]\backslash E_{\epsilon },\hfill \\ u_t,\hfill & t\in E_{\epsilon }.\hfill \end{array}\right.$$

(13)

Here, $E_{\epsilon }$ is any subset of $[0,T]$ of Lebesgue measure $\epsilon$. Clearly, ${\overline{u}}_{·}^{\epsilon ,1}\in {\mathcal{U}}^1$. When player 1 plays the spike-perturbed control ${\overline{u}}_{·}^{\epsilon ,1}$ and player 2 plays the equilibrium control ${\widehat{u}}_{·}^2$, we denote the dynamics by

$$\left\{\begin{array}{cc}d{\overline{Y}}_t^{\epsilon ,1}=b^1(t,{\overline{\Theta }}_t^{\epsilon ,1},{\overline{Y}}_t^{\epsilon ,2},\mathcal{L}({\overline{Y}}_t^{\epsilon ,2}),{\widehat{u}}_t^2,{\overline{Z}}_t^{\epsilon })dt+{\overline{Z}}_t^{\epsilon ,1,1}d{W}_t^1+{\overline{Z}}_t^{\epsilon ,1,2}d{W}_t^2,\hfill & {\overline{Y}}_T^1=y^1,\hfill \\ d{\overline{Y}}_t^{\epsilon ,2}=b^2(t,{\overline{Y}}_t^{\epsilon ,2},\mathcal{L}({\overline{Y}}_t^{\epsilon ,2}),{\widehat{u}}_t^2,{\overline{\Theta }}_t^{\epsilon ,1},{\overline{Z}}_t^{\epsilon })dt+{\overline{Z}}_t^{\epsilon ,2,1}d{W}_t^1+{\overline{Z}}_t^{\epsilon ,2,2}d{W}_t^2,\hfill & {\overline{Y}}_T^2=y^2.\hfill \end{array}\right.$$

(14)

The performance of the perturbed dynamics (14) will be compared with that of the equilibrium dynamics

$$\left\{\begin{array}{cc}\hfill d{\widehat{Y}}_t^1& =b^1(t,{\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t)dt+{\widehat{Z}}_t^{1,1}d{W}_t^1+{\widehat{Z}}_t^{1,2}d{W}_t^2,{\widehat{Y}}_T^1=y^1,\hfill \\ \hfill d{\widehat{Y}}_t^2& =b^2(t,{\widehat{\Theta }}_t^2,{\widehat{\Theta }}_t^1,{\widehat{Z}}_t)dt+{\widehat{Z}}_t^{2,1}d{W}_t^1+{\widehat{Z}}_t^{2,2}d{W}_t^2,{\widehat{Y}}_T^2=y^2.\hfill \end{array}\right.$$

(15)

For simplicity, we write for $\phi \in \{b^1,f^1,h^1\}$, $\psi \in \{b^2,f^2,h^2\}$, $\vartheta \in \{b^i,f^i,h^i,i=1,2\}$,

$$\begin{array}{cc}\hfill {\overline{\phi }}_t^{\epsilon }& :=\phi (t,{\overline{\Theta }}_t^{\epsilon ,1},{\overline{Y}}_t^{\epsilon ,2},\mathcal{L}({\overline{Y}}_t^{\epsilon ,2}),{\widehat{u}}_t^2,{\overline{Z}}_t^{\epsilon }),\hfill \\ \hfill {\overline{\psi }}_t^{\epsilon }& :=\psi (t,{\overline{Y}}_t^{\epsilon ,2},\mathcal{L}({\overline{Y}}_t^{\epsilon ,2}),{\widehat{u}}_t^2,{\overline{\Theta }}_t^{\epsilon ,1},{\overline{Z}}_t^{\epsilon }),\hfill \\ \hfill {\widehat{\vartheta }}_t& :=\vartheta (t,{\widehat{\Theta }}_t^i,{\widehat{\Theta }}_t^{-i},{\widehat{Z}}_t).\hfill \end{array}$$

(16)

In this shorthand notation, which will be used from now on, the difference in performance is

$$J^1({\overline{u}}_{·}^{\epsilon ,1};{\widehat{u}}_{·}^2)-J^1({\widehat{u}}_{·}^1;{\widehat{u}}_{·}^2)=\mathbb{E}\left[{\int }_0^T{\overline{f}}_t^{\epsilon ,1}-{\widehat{f}}_t^{1}dt+{\overline{h}}_0^{\epsilon ,1}-{\widehat{h}}_0^1\right].$$

(17)

Any derivative of $f:a\to f(a)$ will be denoted ${\partial }_{a}f$, indifferent of the space the function is mapping from/to.

Assumption 4.

The functions

$$\begin{array}{cc}\hfill (y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2,z)& \mapsto b^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z)\hfill \\ \hfill (y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2)& \mapsto f^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i})\hfill \\ \hfill (y^1,{\mu }^1,y^2,{\mu }^2)& \mapsto h^i(y^i,{\mu }^i,y^{-i},{\mu }^{-i})\hfill \end{array}$$

(18)

are for all t a.s. differentiable at $({\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t)$, $({\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2)$ and $({\widehat{Y}}_0^1,\mathcal{L}({\widehat{Y}}_0^1),{\widehat{Y}}_0^2,\mathcal{L}({\widehat{Y}}_0^2))$ respectively. Furthermore,

$${\partial }_{y^j}{\widehat{b}}_t^i,{\partial }_{{\mu }^j}{\widehat{b}}_t^i,{\partial }_{y^j}{\widehat{f}}_t^i,{\partial }_{{\mu }^j}{\widehat{f}}_t^i,i,j=1,2,$$

(19)

are for all t a.s. uniformly bounded, and

$${\partial }_{y^j}{\widehat{h}}_0^i+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0^i)\right]\in L_{{\mathcal{F}}_0}^2(\mathrm{\Omega };{\mathbb{R}}^d).$$

(20)

For $i=1,2$,

$$\begin{array}{cc}\hfill {\overline{h}}_0^{\epsilon ,i}-{\widehat{h}}_0^i& =\sum _{j=1}^2\{{\partial }_{y^j}{\widehat{h}}_0^i({\overline{Y}}_0^{\epsilon ,j}-{\widehat{Y}}_0^j)+\mathbb{E}[{({\partial }_{{\mu }^j}{\widehat{h}}_0^i)}^{\ast }({\overline{Y}}_0^{\epsilon ,j}-{\widehat{Y}}_0^j)\left]\right\}\hfill \\ & \hfill +\sum _{j=1}^2\{o\left(|{\overline{Y}}_0^{\epsilon ,j}-{\widehat{Y}}_0^j|\right)+o\left(\mathbb{E}[|{\overline{Y}}_0^{\epsilon ,j}-{\widehat{Y}}_0^j{{|}^2]}^{1/2}\right)\}.\end{array}$$

(21)

A brief overview on differentiation of ${\mathcal{P}}_2({\mathbb{R}}^d)$-valued functions is found in Appendix A, and the notation ${({\partial }_{{\mu }^j}{\widehat{h}}_0^i)}^{\ast }$ is defined in (A8). Both ${\overline{Y}}_t^{\epsilon ,1}-{\widehat{Y}}_t^1$ and ${\overline{Y}}_t^{\epsilon ,2}-{\widehat{Y}}_t^2$ appear in (21), this suggests that we need to introduce two first order variation processes. That is, we want $({\tilde{Y}}_{·}^i,[{\tilde{Z}}_{·}^{i,1},{\tilde{Z}}_{·}^{i,2}])$, $i=1,2,$ that for some $C>0$ satisfies

$$\begin{array}{c}\underset{0\le t\le T}{sup}\mathbb{E}\left[|{\tilde{Y}}_t^i{|}^2+\sum _{j=1}^2{\int }_0^t{\parallel {\tilde{Z}}_s^{i,j}\parallel }_F^{2}ds\right]\le C{\epsilon }^2,\hfill \\ \underset{0\le t\le T}{sup}\mathbb{E}\left[|{\overline{Y}}_t^{\epsilon ,i}-{\widehat{Y}}_t^i-{\tilde{Y}}_t^i{|}^2+\sum _{j=1}^2{\int }_0^t{\parallel {\overline{Z}}_s^{\epsilon ,i,j}-{\widehat{Z}}_s^{i,j}-{\tilde{Z}}_s^{i,j}\parallel }_F^{2}ds\right]\le C{\epsilon }^2.\hfill \end{array}$$

(22)

Let ${\delta }_i$ denote variation in $u_{·}^i$ so that for $\vartheta \in \{f^i,b^i,i=1,2\}$,

$${\delta }_i\vartheta (t):=\vartheta (t,{\widehat{Y}}_t^i,\mathcal{L}({\widehat{Y}}_t^i),{\overline{u}}_t^{\epsilon ,i},{\widehat{\Theta }}_t^{-i},{\widehat{Z}}_t)-{\widehat{\vartheta }}_t.$$

(23)

Assumption 5.

For $y^i,{\mu }^i\in S$, $i=1,2$, $z\in {\mathbb{R}}^{d\times (2d_1+2d_2)}$ and $(u^1,u^2),(v^1,v^2)\in U^1\times U^2$, there exists a constant $L>0$ such that

$$\begin{array}{c}|b^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z)-b^i(t,y^i,{\mu }^i,v^i,y^{-i},{\mu }^{-i},v^{-i},z)|\le L\sum _{j=1}^2{d}_{U^j}(u^j,v^j),\hfill \end{array}$$

(24)

a.s. for all $t\in [0,T]$.

Lemma 1.

Let Assumptions 1, 2, 4 and 5 be in force. The first order variation processes that satisfy (22) is given by the following system of BSDEs,

$$\{\begin{array}{cc}\hfill d{\tilde{Y}}_t^i& =\left(\sum _{j=1}^2\{{\partial }_{y^j}{\widehat{b}}_t^i{\tilde{Y}}_t^j+\mathbb{E}\left[{({\partial }_{{\mu }^j}{\widehat{b}}_t^i)}^{\ast }{\tilde{Y}}_t^j\right]\}+\sum _{j,k=1}^2{\partial }_{z^{j,k}}{\widehat{b}}_t^i{\tilde{Z}}_t^{j,k}+{\delta }_1{b}^i(t)1_{E_{\epsilon }}(t)\right)dt\hfill \\ & +\sum _{j=1}^2{\tilde{Z}}_t^{i,j}d{W}_t^j,\hfill \\ \hfill {\tilde{Y}}_T^i& =0,\hfill \\ \hfill i& =1,2.\hfill \end{array}$$

(25)

A proof is found in the appendix. By Lemma 1,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\overline{h}}_0^{\epsilon ,1}-{\widehat{h}}_0^1\right]& =\mathbb{E}\left[\sum _{j=1}^2{\partial }_{y^j}{\widehat{h}}_0^1{\tilde{Y}}_0^j+\mathbb{E}\left[{({\partial }_{{\mu }^j}{\widehat{h}}_0^1)}^{\ast }{\tilde{Y}}_0^j\right]\right]+o(\epsilon )\hfill \\ & =\mathbb{E}\left[\sum _{j=1}^2{p}_0^{1,j}{\tilde{Y}}_0^j\right]+o(\epsilon ),\hfill \end{array}$$

(26)

where the introduced costates $p_{·}^{1,j}$, $j=1,2$, satisfy $p_0^{1,j}:={\partial }_{y^j}{\widehat{h}}_0^1+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0^1)\right]$. The notation ${\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0^1)$ is defined in (A10). Assumption 4 grants us existence and uniqueness to Equation (27) below.

Lemma 2 (Duality relation).

Lett Assumptions 1, 2 and 4 hold. Let $p_{·}^{1,j}$ be the solution to the SDE

$$\{\begin{array}{cc}\hfill d{p}_t^{1,j}& =-\{{\partial }_{y^j}{\widehat{H}}_t^1+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{H}}_t^1)\right]\}dt-\sum _{k=1}^2{\partial }_{z^{j,k}}{\widehat{H}}_t^{1}d{W}_t^k,\hfill \\ \hfill p_0^{1,j}& ={\partial }_{y^j}{\widehat{h}}_0^1+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0^1)\right],\hfill \end{array}$$

(27)

where for $(y^i,{\mu }^i)\in S$, $i=1,2$, and $(u^1,u^2,z)\in U^1\times U^2\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$,

$$\begin{array}{c}{H}^1(\omega ,t,y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2,z,p_t^{1,1},p_t^{1,2})\hfill \\ :=\sum _{j=1}^2{b}^j(\omega ,t,y^j,{\mu }^j,u^j,y^{-j},{\mu }^{-j},u^{-j},z)p_t^{1,j}-f^1(t,y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2).\hfill \end{array}$$

(28)

Then the following duality relation holds,

$$\mathbb{E}\left[{\sum }_{j=1}^2{p}_0^{1,j}{\tilde{Y}}_0^j\right]=-\mathbb{E}\left[{\int }_0^T{\sum }_{j=1}^2{p}_t^{1,j}{\delta }_1{b}^j(t)1_{E_{\epsilon }}(t)+{\tilde{Y}}_t^j\left({\partial }_{y^j}{\widehat{f}}_t^1+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{f}}_t^1)\right]\right)dt\right].$$

(29)

A proof of the lemma above is found in the appendix. We have that

$$\begin{array}{cc}\hfill {\overline{f}}_t^{\epsilon ,i}-{\widehat{f}}_t^i& =\sum _{j=1}^2\{{\partial }_{y^j}{\widehat{f}}_t^i({\overline{Y}}_t^{\epsilon ,j}-{\widehat{Y}}_t^j)+\mathbb{E}\left[({{\partial }_{{\mu }^j}}^{{\widehat{f}}_t^i})\ast ({\overline{Y}}_t^{\epsilon ,j}-{\widehat{Y}}_t^j)\right]\}+{\delta }_1{f}^i(t)1_{E_{\epsilon }}(t)\hfill \\ & +\sum _{j=1}^2\{o\left(|{\overline{Y}}_t^{\epsilon ,j}-{\widehat{Y}}_t^j|\right)+o\left(\mathbb{E}[|{\overline{Y}}_t^{\epsilon ,j}-{\widehat{Y}}_t^j{{|}^2]}^{1/2}\right)\}.\hfill \end{array}$$

(30)

By the expansion (30) and Lemma 1,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\int }_0^T{\overline{f}}_t^{\epsilon ,1}-{\widehat{f}}_t^{1}dt\right]& =\mathbb{E}\left[{\int }_0^T\sum _{j=1}^2{\tilde{Y}}_t^j\left({\partial }_{y^j}{\widehat{f}}_t^1+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{f}}_t^1)\right]\right)+{\delta }_1{f}^1(t)1_{E_{\epsilon }}(t)dt\right]\hfill \\ & +o(\epsilon ),\hfill \end{array}$$

(31)

which yields

$$\begin{array}{cc}\hfill J^1({\overline{u}}_{·}^{\epsilon ,1};{\widehat{u}}_{·}^2)-J^1({\widehat{u}}_{·}^1;{\widehat{u}}_{·}^2)& =\mathbb{E}\left[{\int }_0^T\{-p_t^{1,1}{\delta }_1{b}^1(t)-p_t^{1,2}{\delta }_1{b}^2(t)+{\delta }_1{f}^1(t)\}{1}_{E_{\epsilon }}(t)dt\right]\hfill \\ & +o(\epsilon ).\hfill \end{array}$$

(32)

Therefore

$$J^1({\overline{u}}_{·}^{\epsilon ,1};{\widehat{u}}_{·}^2)-J^1({\widehat{u}}_{·}^1;{\widehat{u}}_{·}^2)=-\mathbb{E}\left[{\int }_0^T{\delta }_1{H}^1(t)1_{E_{\epsilon }}(t)dt\right]+o(\epsilon ).$$

(33)

From the last identity, we can derive necessary and sufficient conditions for player 1’s best response to ${\widehat{u}}_{·}^2$.

The same argument can be carried out for players 2’s best response to ${\widehat{u}}_{·}^1$. Naturally, we need to impose the corresponding assumptions on player 2’s control. For completeness and later reference, we state now the second player’s version of Lemma 2.

Lemma 3 (Duality relation, player 2).

Let Assumptions 1, 2 and 4 hold, and let $p_{·}^{2,j}$ be the solution to the SDE

$$\{\begin{array}{cc}\hfill d{p}_t^{2,j}& =-\{{\partial }_{y^j}{\widehat{H}}_t^2+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{H}}_t^2)\right]\}dt-\sum _{k=1}^2{\partial }_{z^{j,k}}{\widehat{H}}_t^{2}d{W}_t^k,\hfill \\ \hfill p_0^{2,j}& ={\partial }_{y^j}{\widehat{h}}_0^2+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0^2)\right],\hfill \end{array}$$

(34)

where for $(y^i,{\mu }^i)\in S$, $i=1,2$, and $(u^1,u^2,z)\in U^1\times U^2\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$,

$$\begin{array}{c}{H}^2(\omega ,t,y^2,{\mu }^2,u^2,y^1,{\mu }^1,u^1,z,p_t^{2,1},p_t^{2,2})\hfill \\ :=\sum _{j=1}^2{b}^j(\omega ,t,y^j,{\mu }^j,u^j,y^{-j},{\mu }^{-j},u^{-j},z)p_t^{2,j}-f^2(t,y^2,{\mu }^2,u^2,y^1,{\mu }^1,u^1).\hfill \end{array}$$

(35)

Then the following duality relation holds,

$$\mathbb{E}\left[\sum _{j=1}^2{p}_0^{2,j}{\tilde{Y}}_0^j\right]=-\mathbb{E}\left[{\int }_0^T\sum _{j=1}^2{p}_t^{2,j}{\delta }_2{b}^j(t)1_{E_{\epsilon }}(t)+{\tilde{Y}}_t^j\left({\partial }_{y^j}{\widehat{f}}_t^2+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{y^j}{\widehat{f}}_t^2)\right]dt\right)\right].$$

(36)

Necessary equilibrium conditions can be stated as a system of 6 Equations, 2 state BSDEs and 4 costate (adjoint) SDEs. Sufficient conditions for a Nash equilibrium can now be stated as convexity conditions on the 4 functions $H^i,h^i,i=1,2$. We let Assumptions 1–5 be in place.

Theorem 2.

[Necessary equilibrium conditions] Suppose that $({\widehat{Y}}_{·}^i,[{\widehat{Z}}_{·}^{i,1},{\widehat{Z}}_{·}^{i,2}],{\widehat{u}}_{·}^i)$, $i=1,2$, is an equilibrium for the MFTG and that $p_{·}^{i,j}$, $i,j=1,2$, solve (27) and (34). Then, for $i=1,2$,

$${\widehat{u}}_t^i=\underset{\alpha \in U^i}{\mathit{argmax}}{H}^i(t,{\widehat{Y}}_t^i,\mathcal{L}({\widehat{Y}}_t^i),\alpha ,{\widehat{\Theta }}_t^{-i},{\widehat{Z}}_t,p_t^{i,1},p_t^{i,2}),a.e.t,a.s.$$

(37)

Proof. 

Let $E_{\epsilon }:=[s,s+\epsilon ]$, $u_{·}\in {\mathcal{U}}^1$ and $A\in {\mathcal{F}}_t$ for $t\in E_{ϵ}$. Consider the spike-perturbation

$$u_t^{\epsilon }:=\{\begin{array}{cc}{u}_t{1}_A+{\widehat{u}}_t^1{1}_{A^c},\hfill & t\in E_{\epsilon },\hfill \\ {\widehat{u}}_t^1,\hfill & t\in [0,T]\backslash E_{\epsilon }.\hfill \end{array}$$

(38)

Then

$$\begin{array}{c}{\widehat{H}}_t^1-H^1(t,{\widehat{Y}}_t^1,\mathcal{L}({\widehat{Y}}_t^1),u_t^{\epsilon },{\widehat{\Theta }}_t^2,{\widehat{Z}}_t,p_t^{1,1},p_t^{1,2})=\hfill \\ \left({\widehat{H}}_t^1-H^1(t,{\widehat{Y}}_t^1,\mathcal{L}({\widehat{Y}}_t^1),u_t,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t,p_t^{1,1},p_t^{1,2})\right)1_A{1}_{E_{\epsilon }}(t).\hfill \end{array}$$

(39)

Applying (33), we obtain

$$\frac{1}{\epsilon }\mathbb{E}\left[{\int }_s^{s+\epsilon }\left({\widehat{H}}_t^1-H^1(t,{\widehat{Y}}_t^1,\mathcal{L}({\widehat{Y}}_t^1),u_t,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t,p_t^{1,1},p_t^{1,2})\right)1_{A}dt\right]\ge \frac{1}{\epsilon }o(\epsilon ).$$

(40)

Sending ε to zero yields

$$\mathbb{E}\left[\left({\widehat{H}}_s^1-H^1(s,{\widehat{Y}}_s^1,\mathcal{L}({\widehat{Y}}_s^1),u_s,{\widehat{\Theta }}_s^2,{\widehat{Z}}_s,p_s^{1,1},p_s^{1,2})\right)1_A\right]\ge 0,a.e.s\in [0,T].$$

(41)

The last inequality holds for all $A\in {\mathcal{F}}_s$, thus

$$\mathbb{E}\left[\left({\widehat{H}}_s^1-H^1(s,{\widehat{Y}}_s^1,\mathcal{L}({\widehat{Y}}_s^1),u_s,{\widehat{\Theta }}_s^1,{\widehat{Z}}_s,p_s^{1,1},p_s^{1,2})\right)|{\mathcal{F}}_s\right]\ge 0,a.e.s\in [0,T],a.s.$$

(42)

By measurability of the integrand in (42),

$${\widehat{u}}_t^1=\underset{\alpha \in U^1}{\mathrm{argmax}}{H}^1(t,{\widehat{Y}}_t^1,\mathcal{L}({\widehat{Y}}_t^1),\alpha ,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t,p_t^{1,1},p_t^{1,2}),a.e.t\in [0,T],a.s.$$

(43)

The same argument yields

$${\widehat{u}}_t^2=\underset{\alpha \in U^2}{\mathrm{argmax}}{H}^2(t,{\widehat{Y}}_t^2,\mathcal{L}({\widehat{Y}}_t^2),\alpha ,{\widehat{\Theta }}_t^1,{\widehat{Z}}_t,p_t^{2,1},p_t^{2,2}),a.e.t\in [0,T],a.s.$$

(44)

□

Theorem 3.

[Sufficient equilibrium conditions] Suppose that ${\widehat{u}}_{·}^1$ and ${\widehat{u}}_{·}^2$ satisfy (37). Suppose furthermore that for $(t,p^{i,1},p^{i,2},z)\in [0,T]\times {\mathbb{R}}^d\times {\mathbb{R}}^d\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$, $i=1,2$,

$$(y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2)\mapsto H^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z,p^{i,1},p^{i,2})$$

(45)

is concave a.s. and

$$(y^1,{\mu }^1,y^2,{\mu }^2)\mapsto h^i(y^i,{\mu }^i,y^{-i},{\mu }^{-i})$$

(46)

is convex a.s. Then ${\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2$ constitute an equilibrium control and $({\widehat{Y}}_{·}^i,[{\widehat{Z}}_{·}^{i,1},{\widehat{Z}}_{·}^{i,2}],{\widehat{u}}_{·}^i)$, $i=1,2$, is an equilibrium for the MFTG.

Proof. 

By assumption, ${\delta }_i{H}^i(t)\le 0$ for any spike variation, almost surely for a.e. t. Applying the convexity and concavity assumptions in the expansion steps results in the inequality

$$0\le -\mathbb{E}\left[{\int }_0^T{\delta }_i{H}^i(t)1_{E_{\epsilon }}(t)dt\right]\le J^i(u_{·}^i;{\widehat{u}}_{·}^{-i})-J^i({\widehat{u}}_{·}^i;{\widehat{u}}_{·}^{-i}).$$

(47)

□

4. Problem 2: MFTC

Carrying out a similar argument to that of the previous section, we find necessary optimality conditions for problem (10). Also, we readily get a verification theorem. The pair $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$ is optimal if

$$J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\le J(u_{·}^1,u_{·}^2),(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2.$$

(48)

Assume from now on that $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$ is an optimal control. We study the inequality (48) when $({\check{u}}_{·}^{\epsilon ,1},{\check{u}}_{·}^{\epsilon ,2})$ is a spike-perturbation of $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$,

$$({\check{u}}_t^{\epsilon ,1},{\check{u}}_t^{\epsilon ,2}):=\{\begin{array}{cc}({\widehat{u}}_t^1,{\widehat{u}}_t^2),\hfill & t\in [0,T]\backslash E_{\epsilon },\hfill \\ (u_t^1,u_t^2),\hfill & t\in E_{\epsilon },\hfill \end{array}$$

(49)

where $E_{\epsilon }$ is any subset of $[0,T]$ of Lebesgue measure ε and $(u_{·}^1,u_{·}^2)\in {\mathcal{U}}^1\times {\mathcal{U}}^2$. When the players use the perturbed control, we denote the state dynamics by

$$\{\begin{array}{cc}\hfill d{\check{Y}}_t^{\epsilon ,1}& =b^1(t,{\check{\Theta }}_t^{\epsilon ,1},{\check{\Theta }}_t^{\epsilon ,2},{\check{Z}}_t^{\epsilon })dt+{\check{Z}}_t^{\epsilon ,1,1}d{W}_t^1+{\check{Z}}_t^{\epsilon ,1,2}d{W}_t^2,{\check{Y}}_T^1=y^1,\hfill \\ \hfill d{\check{Y}}_t^{\epsilon ,2}& =b^2(t,{\check{\Theta }}_t^{\epsilon ,2},{\check{\Theta }}_t^{\epsilon ,1},{\check{Z}}_t^{\epsilon })dt+{\check{Z}}_t^{\epsilon ,2,1}d{W}_t^1+{\check{Z}}_t^{\epsilon ,2,2}d{W}_t^2,{\check{Y}}_T^2=y^2,\hfill \end{array}$$

(50)

and we will compare their performance to that of the optimally controlled state dynamics

$$\{\begin{array}{cc}\hfill d{\widehat{Y}}_t^1& =b^1(t,{\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t)dt+{\widehat{Z}}_t^{1,1}d{W}_t^1+{\widehat{Z}}_t^{1,2}d{W}_t^2,{\widehat{Y}}_T^1=y^1,\hfill \\ \hfill d{\widehat{Y}}_t^2& =b^2(t,{\widehat{\Theta }}_t^2,{\widehat{\Theta }}_t^1,{\widehat{Z}}_t)dt+{\widehat{Z}}_t^{2,1}d{W}_t^1+{\widehat{Z}}_t^{2,2}d{W}_t^2,{\widehat{Y}}_T^2=y^2.\hfill \end{array}$$

(51)

For simplicity, we write for $\vartheta \in \{b^i,f^i,h^i,i=1,2\}$,

$$\begin{array}{cc}\hfill {\check{\vartheta }}_t^{\epsilon }& :=\vartheta (t,{\check{\Theta }}_t^{\epsilon ,i},{\check{\Theta }}_t^{\epsilon ,-i},{\check{Z}}_t^{\epsilon }),\hfill \\ \hfill {\widehat{\vartheta }}_t& :=\vartheta (t,{\widehat{\Theta }}_t^i,{\widehat{\Theta }}_t^{-i},{\widehat{Z}}_t),\hfill \end{array}$$

(52)

and in this notation,

$$\begin{array}{cc}\hfill J({\check{u}}_{·}^{\epsilon ,1},{\check{u}}_{·}^{\epsilon ,2})-J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)& =\mathbb{E}\left[{\int }_0^T{\check{f}}_t^{\epsilon ,1}+{\check{f}}_t^{\epsilon ,2}-{\widehat{f}}_t^1-{\widehat{f}}_t^{2}dt+{\check{h}}_0^{\epsilon ,1}+{\check{h}}_0^{\epsilon ,2}-{\widehat{h}}_0^1-{\widehat{h}}_0^2\right]\hfill \\ & =\mathbb{E}\left[{\int }_0^T{\check{f}}_t^{\epsilon }-{\widehat{f}}_{t}dt+{\check{h}}_0^{\epsilon }-{\widehat{h}}_0\right]\hfill \end{array}$$

(53)

where $f_t:=f_t^1+f_t^2$ and $h_t:=h_t^1+h_t^2$. Again, we want to find first order variation processes $({\tilde{Y}}_{·}^i,[{\tilde{Z}}_{·}^{i,1},{\tilde{Z}}_{·}^{i,2}])$, $i=1,2,$ that satisfy (22) with $({\overline{Y}}_{·}^{\epsilon ,i},[{\overline{Z}}_{·}^{\epsilon ,i,1},{\overline{Z}}_{·}^{\epsilon ,i,2}])$ replaced by its ’checked’ counterpart $({\check{Y}}_{·}^{\epsilon ,i},[{\check{Z}}_{·}^{\epsilon ,i,1},{\check{Z}}_{·}^{\epsilon ,i,2}])$.

Assumption 6.

The functions

$$\begin{array}{cc}\hfill (y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2,z)& \mapsto b^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i},z)\hfill \\ \hfill (y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2)& \mapsto f^i(t,y^i,{\mu }^i,u^i,y^{-i},{\mu }^{-i},u^{-i})\hfill \\ \hfill (y^1,{\mu }^1,y^2,{\mu }^2)& \mapsto h^i(y^i,{\mu }^i,y^{-i},{\mu }^{-i})\hfill \end{array}$$

(54)

are for all t a.s. differentiable at $({\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2,{\widehat{Z}}_t)$, $({\widehat{\Theta }}_t^1,{\widehat{\Theta }}_t^2)$ and $({\widehat{Y}}_0^1,\mathcal{L}({\widehat{Y}}_0^1),{\widehat{Y}}_0^2,\mathcal{L}({\widehat{Y}}_0^2))$ respectively. Furthermore,

$${\partial }_{y^j}{\widehat{b}}_t^i,{\partial }_{{\mu }^j}{\widehat{b}}_t^i,{\partial }_{y^j}{\widehat{f}}_t^i,{\partial }_{{\mu }^j}{\widehat{f}}_t^i,i,j=1,2,$$

(55)

are for all t a.s. uniformly bounded and ${\partial }_{y^j}{\widehat{h}}_0^i+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{\mu }^j{\widehat{h}}_0^i)\right]\in L_{{\mathcal{F}}_0}^2(\mathrm{\Omega };{\mathbb{R}}^d)$.

Notice that the point of differentiability is generally not the same in Assumptions 4 and 6. Above, $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$ is an optimal control while in Assumption 4, it is an equilibrium control. Let δ denote simultaneous variation in controls, for $\vartheta \in \{f^i,b^i,i=1,2\}$,

$$\delta \vartheta (t):={\delta }_1\vartheta (t)+{\delta }_2\vartheta (t).$$

(56)

Lemma 4.

Let Assumptions 1, 2, 5 and 6 be in force. The first order variation processes that satisfy the ’checked’ version of (22) is given by the following system of BSDEs,

$$\{\begin{array}{cc}\hfill d{\tilde{Y}}_t^i& =\left(\sum _{j=1}^2\{{\partial }_{y^j}{\widehat{b}}^i{\tilde{Y}}_t^j+\mathbb{E}\left[{({\partial }_{{\mu }^j}{\widehat{b}}^i)}^{\ast }{\tilde{Y}}_t^j\right]\}+\delta b^i(t)1_{E_{\epsilon }}(t)+\sum _{j,k=1}^2{\partial }_{z^{j,k}}{\widehat{b}}^i{\tilde{Z}}_t^{j,k}\right)dt\hfill \\ & +\sum _{j=1}^2{\tilde{Z}}_t^{i,j}d{W}_t^j,\hfill \\ \hfill {\tilde{Y}}_T^i& =0,\hfill \\ \hfill i& =1,2,\hfill \end{array}$$

(57)

The proof follows the same steps as the proof of Lemma 1.

By Lemma 4,

$$\mathbb{E}\left[{\check{h}}_0^{\epsilon }-{\widehat{h}}_0\right]=\mathbb{E}\left[\sum _{j=1}^2{p}_0^j{\tilde{Y}}_0^j\right]+o(\epsilon ),$$

(58)

where $p_0^j:={\partial }_{y^j}{\widehat{h}}_0+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0)\right]$.

Lemma 5 (Duality relation).

Let Assumptions 1, 2 and 6 hold. Let $p_{·}^j$ be the solution to the SDE

$$\{\begin{array}{cc}\hfill d{p}_t^j& =-\{{\partial }_{y^j}{\widehat{H}}_t+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{H}}_t)\right]\}dt-\sum _{k=1}^2{\partial }_{z^{j,k}}{\widehat{H}}_{t}d{W}_t^k,\hfill \\ \hfill p_0^j& ={\partial }_{y^j}{\widehat{h}}_0+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{h}}_0)\right],\hfill \end{array}$$

(59)

where for $(y^i,{\mu }^i)\in S$, $i=1,2$, and ($u^1,u^2,z)\in U^1\times U^2\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$,

$$\begin{array}{c}H(\omega ,t,y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2,z,p_t^1,p_t^2)\hfill \\ :=\sum _{j=1}^2{b}^j(\omega ,t,y^j,{\mu }^j,u^j,y^{-j},{\mu }^{-j},u^{-j},z)p_t^j-f^j(t,y^j,{\mu }^j,u^j,y^{-j},{\mu }^{-j},u^{-j}).\hfill \end{array}$$

(60)

Then the following duality relation holds,

$$\mathbb{E}\left[\sum _{j=1}^2{p}_0^j{\tilde{Y}}_0^j\right]=-\mathbb{E}\left[{\int }_0^T\sum _{j=1}^2{p}_t^j\delta b^j(t)1_{E_{\epsilon }}(t)+{\tilde{Y}}_t^j\left({\partial }_{y^j}{\widehat{f}}_t+\mathbb{E}\left[{\phantom{(}}^{\ast }({\partial }_{{\mu }^j}{\widehat{f}}_t)\right]\right)dt\right].$$

(61)

The proof of Lemma 5 is almost identical that of Lemma 2. By Lemma 4,

$$J({\check{u}}_{·}^{\epsilon ,1},{\check{u}}_{·}^{\epsilon ,2})-J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)=\mathbb{E}\left[{\int }_0^T\{-p_t^1\delta b^1(t)-p_t^2\delta b^2(t)+\delta f(t)\}{1}_{E_{\epsilon }}(t)dt\right]+o(\epsilon ).$$

(62)

Thus

$$J({\check{u}}_{·}^{\epsilon ,1},{\check{u}}_{·}^{\epsilon ,2})-J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)=-\mathbb{E}\left[{\int }_0^T\delta H(t)1_{E_{\epsilon }}(t)dt\right]+o(\epsilon ).$$

(63)

In the following two theorems, Assumptions 1–3, 5 and 6 are in force.

Theorem 4 (Necessary optimality conditions).

Suppose that $({\widehat{Y}}_{·}^i,[{\widehat{Z}}_{·}^{i,1},{\widehat{Z}}_{·}^{i,2}])$, $i=1,2$ is an optimal solution to the MFTC and that $p_{·}^i$, $i=1,2$, solves (59). Then, for $i=1,2$,

$$({\widehat{u}}_t^1,{\widehat{u}}_t^2)=\underset{(v,w)\in U^1\times U^2}{\mathit{argmax}}H(t,{\widehat{Y}}_t^1,\mathcal{L}({\widehat{Y}}_t^1),v,{\widehat{Y}}_t^2,\mathcal{L}({\widehat{Y}}_t^2),w,{\widehat{Z}}_t,p_t^1,p_t^2),a.e.t,a.s.$$

(64)

Theorem 5 (Sufficient optimality conditions).

Suppose $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$ satisfy (64). Suppose furthermore that for $(t,p^1,p^2,z)\in [0,T]\times {\mathbb{R}}^d\times {\mathbb{R}}^d\times {\mathbb{R}}^{d\times (2d_1+2d_2)}$, $i=1,2$,

$$(y^1,{\mu }^1,u^1,y^2,{\mu }^2,u^2)\mapsto H(t,y^1,{\mu }^1,u^1,y^2,{\mu }^{2}s,u^2,z,p^1,p^2)$$

(65)

is concave a.s. and

$$(y^1,{\mu }^1,y^2,{\mu }^2)\mapsto h(y^1,{\mu }^1,y^2,{\mu }^2)$$

(66)

is convex a.s. Then $({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)$ is an optimal control and $({\widehat{Y}}_{·}^i,[{\widehat{Z}}_{·}^{i,1},{\widehat{Z}}_{·}^{i,2}],{\widehat{u}}_{·}^i)$, $i=1,2$ solves the MFTC.

5. Example: The Linear-Quadratic Case

In this section we consider a linear-quadratic version of (9) and (10), in the one dimensional case. Let $a_i,c_{i,j},q_{i,j},{\overline{q}}_{i,j},{\tilde{q}}_{i,j},{\overline{s}}_{i,j},s_i,{\overline{s}}_i^E,r_i:[0,T]\mapsto \mathbb{R}$, $i,j=1,2$ be deterministic coefficient functions, uniformly bounded over $[0,T]$. Additionally, $r_i(t)\ge ϵ>0$ for $i=1,2$. Define

$$\begin{array}{cc}\hfill b^i(t,{\Theta }_t^i,{\Theta }_t^{-i},Z_t)& =a_i(t)u_t^i+\sum _{j=1}^2{c}_{i,j}(t)W_t^j,\hfill \\ \hfill f^i(t,{\Theta }_t^i,{\Theta }_t^{-i})& =\sum _{j=1}^2\{\frac{1}{2}{q}_{i,j}(t){(Y_t^j)}^2+\frac{1}{2}{\overline{q}}_{i,j}(t)\mathbb{E}{[Y_t^j]}^2+{\tilde{q}}_{i,j}(t)Y_t^j\mathbb{E}[Y_t^j]\hfill \\ & +{\overline{s}}_{i,j}(t)\mathbb{E}[Y_t^j]Y_t^{-j}\}\hfill \\ & +s_i(t)Y_t^i{Y}_t^{-i}+{\overline{s}}_i^E(t)\mathbb{E}[Y_t^i]\mathbb{E}[Y_t^{-i}]+\frac{1}{2}{r}_i(t){(u_t^i)}^2.\hfill \end{array}$$

(67)

The uniform boundedness of the coefficients implies Assumptions 1–6, given initial costs $h^1,h^2$, satisfying Assumption 4 and 6. Assumption 3 (integrability of $f^i$) follows by classical BSDE estimates [38]. Recall player 1 and 2’s Hamiltonian, defined in (28) and (35). In the setup of this example,

$$\begin{array}{c}{H}^i(t,{\Theta }_t^i,{\Theta }_t^{-i},Z_t)=\hfill \\ \left(a_1(t)u_t^1+\sum _{j=1}^2{c}_{1,j}{W}_t^j\right)p^{i,1}+\left(a_2(t)u_t^2+\sum _{j=1}^2{c}_{2,j}{W}_t^j\right)p^{i,2}\hfill \\ -\sum _{j=1}^2\{\frac{1}{2}{q}_{i,j}(t){(Y_t^j)}^2+\frac{1}{2}{\overline{q}}_{i,j}(t)\mathbb{E}{[Y_t^j]}^2+{\tilde{q}}_{i,j}(t)Y_t^j\mathbb{E}[Y_t^j]+{\overline{s}}_{i,j}(t)\mathbb{E}[Y_t^j]Y_t^{-j}\}\hfill \\ -s_i(t)Y_t^i{Y}_t^{-i}-{\overline{s}}_i^E(t)\mathbb{E}[Y_t^i]\mathbb{E}[Y_t^{-i}]-\frac{1}{2}{r}_i(t){(u_t^i)}^2.\hfill \end{array}$$

(68)

The Hessian of $(y^1,\cdots ,u^2)\mapsto H^1(t,y^1,\cdots ,u^2,z,p^{1,1},p^{1,2})$ is

$${\mathcal{H}}^1(t):=-\left[\begin{array}{cccccc}{q}_{1,1}(t)& {\tilde{q}}_{1,1}(t)& 0& s_1(t)& {\overline{s}}_{1,2}(t)& 0\\ {\tilde{q}}_{1,1}(t)& {\overline{q}}_{1,1}(t)& 0& {\overline{s}}_{1,1}(t)& {\overline{s}}_1^E(t)& 0\\ 0& 0& r_1(t)& 0& 0& 0\\ s_1(t)& {\overline{s}}_{1,1}(t)& 0& q_{1,2}(t)& {\tilde{q}}_{1,2}(t)& 0\\ {\overline{s}}_{1,2}(t)& {\overline{s}}_1^E(t)& 0& {\tilde{q}}_{1,2}(t)& {\overline{q}}_{1,2}(t)& 0\\ 0& 0& 0& 0& 0& 0\end{array}\right]$$

(69)

and the Hessian of $(y^1,\cdots ,u^2)\mapsto H^2(t,y^1,\cdots ,u^2,z,p^{2,1},p^{2,2})$ is

$${\mathcal{H}}^2(t):=-\left[\begin{array}{cccccc}{q}_{2,1}(t)& {\tilde{q}}_{2,1}(t)& 0& s_2(t)& {\overline{s}}_{2,2}(t)& 0\\ {\tilde{q}}_{2,1}(t)& {\overline{q}}_{2,1}(t)& 0& {\overline{s}}_{2,1}(t)& {\overline{s}}_2^E(t)& 0\\ 0& 0& 0& 0& 0& 0\\ s_2(t)& {\overline{s}}_{2,1}(t)& 0& q_{2,2}(t)& {\tilde{q}}_{2,2}(t)& 0\\ {\overline{s}}_{2,2}(t)& {\overline{s}}_2^E(t)& 0& {\tilde{q}}_{2,2}(t)& {\overline{q}}_{2,2}(t)& 0\\ 0& 0& 0& 0& 0& r_2(t)\end{array}\right].$$

(70)

The coefficients are further assumed to be such that ${\mathcal{H}}^1(t)$ and ${\mathcal{H}}^2(t)$ are negative semi-definite for all $t\in [0,T]$. Also, we assume that $(y^1,\cdots ,{\mu }^2)\mapsto h^i(y^1,\cdots ,{\mu }^2)$, yet unspecified, is convex. Theorem 3 yields

$${\widehat{u}}_t^i=a_i(t)r_i^{-1}(t)p_t^{i,i},$$

(71)

where $p_{·}^{i,i}$ solves (27) or (34), depending on i. In fact the equilibrium is unique in this case, since ${\widehat{u}}_{·}^i$ is the unique pointwise solution to (37) and $p_{·}^{i,i}$ is unique, see (A25) and (A26). By Theorem 5,

$${\widehat{u}}_t^i=a_i(t)r_i^{-1}(t)p_t^i,$$

(72)

where $p_{·}^i$ solves (59), is an optimal control for the linear-quadratic MFTC and it is unique.

5.1. MFTG

The equilibrium dynamics are

$$\{\begin{array}{cc}\hfill d{\widehat{Y}}_t^i& =\left(a_i^2(t)r_i^{-1}(t)p_t^{i,i}+\sum _{j=1}^2{c}_{i,j}{W}_t^j\right)dt+{\widehat{Z}}_t^{i,1}d{W}_t^1+{\widehat{Z}}_t^{i,2}d{W}_t^2,\hfill \\ \hfill {\widehat{Y}}_T^i& =y_T^i.\hfill \end{array}$$

(73)

We see that only two costate processes, $p_{·}^{1,1}$ and $p_{·}^{2,2}$, are relevant here. This is a consequence of the lack of explicit dependence on $u^{-i}$ in the $b^i$ and $f^i$ specified in (67). Nevertheless, the running cost $f^i$ depends implicitly on $u^{-i}$ through player $-i$’s state and mean.

We make the following ansatz: there exists deterministic functions ${\alpha }_i,{\overline{\alpha }}_i,{\beta }_i,{\overline{\beta }}_i,{\theta }_i:[0,T]\to \mathbb{R}$, $i=1,2$ and ${\gamma }_{i,j}:[0,T]\to \mathbb{R}$, $i,j=1,2$, such that

$${\widehat{Y}}_t^i={\alpha }_i(t)p_t^{i,i}+{\overline{\alpha }}_i(t)\mathbb{E}[p_t^{i,i}]+{\beta }_i(t)p_t^{-i,-i}+{\overline{\beta }}_i(t)\mathbb{E}[p_t^{-i,-i}]+{\gamma }_{i,1}(t)W_t^1+{\gamma }_{i,2}(t)W_t^2+{\theta }_i(t).$$

(74)

Clearly, we need to impose the terminal conditions

$${\alpha }_i(T)=0,{\overline{\alpha }}_i(T)=0,{\beta }_i(T)=0,{\overline{\beta }}_i(T)=0,{\gamma }_{i,j}(T)=0,{\theta }_i(T)=y_T^i.$$

(75)

Calculations presented in the appendix identifies coefficients and yields the following system of ODEs determining ${\alpha }_i(·),\cdots ,{\theta }_i(·)$,

$$\{\begin{array}{c}{\dot{\alpha }}_i(t)+{\alpha }_i(t)P^i(t)+{\beta }_i(t)R^{-i}(t)=a_i^2(t)r_i^{-1}(t),\hfill \\ {\dot{\overline{\alpha }}}_i(t)+{\alpha }_i(t){\overline{P}}_i(t)+{\overline{\alpha }}_i(t)(P^i(t)+{\overline{P}}^i(t))+{\beta }_i(t){\overline{R}}^{-i}(t)+{\overline{\beta }}_i(t)(R^{-i}(t)+{\overline{R}}^{-i}(t))=0,\hfill \\ {\dot{\beta }}_i(t)+{\alpha }_i(t)R^i(t)+{\beta }_i(t)P^{-i}(t)=0,\hfill \\ {\dot{\overline{\beta }}}_i(t)+{\alpha }_i(t){\overline{R}}^i(t)+{\overline{\alpha }}_i(t)(R^i(t)+{\overline{R}}^i(t))+{\beta }_i(t){\overline{P}}^{-i}(t)+{\overline{\beta }}_i(t)(P^{-i}(t)+{\overline{P}}^{-i}(t))=0,\hfill \\ {\dot{\gamma }}_{i,1}(t)+{\alpha }_i(t){\mathrm{\Phi }}^i(t)+{\beta }_i(t){\mathrm{\Phi }}^{-i}(t)=c_{i,1}(t),\hfill \\ {\dot{\gamma }}_{i,2}(t)+{\alpha }_i(t){\mathrm{\Psi }}^i(t)+{\beta }_i(t){\mathrm{\Psi }}^{-i}(t)=c_{i,2}(t),\hfill \\ {\dot{\theta }}_i(t)+{\theta }_i(t)\left(({\alpha }_i(t)+{\overline{\alpha }}_i(t))(Q_i(t)+{\overline{Q}}_i(t))+({\beta }_i(t)+{\overline{\beta }}_i(t))(S_{-i}(t)+{\overline{S}}_{-i}(t))\right)\hfill \\ +{\theta }_{-i}(t)\left(({\alpha }_i(t)+{\overline{\alpha }}_i(t))(S_i(t)+{\overline{S}}_i(t))+({\beta }_i(t)+{\overline{\beta }}_i(t))(Q_{-i}(t)+{\overline{Q}}_{-i}(t))\right)=0,\hfill \end{array}$$

(76)

where

$$\begin{array}{c}{P}^i(t):=Q_i(t){\alpha }_i(t)+S_i(t){\beta }_{-i}(t),\hfill \\ {\overline{P}}^i(t):=Q_i(t){\overline{\alpha }}_i(t)+{\overline{Q}}_i(t)({\alpha }_i(t)+{\overline{\alpha }}_i(t))+S_i(t){\overline{\beta }}_{-i}(t)+{\overline{S}}_i(t)({\beta }_{-i}(t)+{\overline{\beta }}_{-i}(t)),\hfill \\ R^i(t):=Q_i(t){\beta }_i(t)+S_i(t){\alpha }_{-i}(t),\hfill \\ {\overline{R}}^i(t):=Q_i(t){\overline{\beta }}_i(t)+{\overline{Q}}_i(t)({\beta }_i(t)+{\overline{\beta }}_i(t))+S_i(t){\overline{\alpha }}_{-i}(t)+{\overline{S}}_i(t)({\alpha }_{-i}(t)+{\overline{\alpha }}_{-i}(t)),\hfill \\ {\mathrm{\Phi }}^i(t):=(Q_i(t){\gamma }_{i,1}(t)+S_i(t){\gamma }_{-i,1}(t)),{\mathrm{\Psi }}^i(t):=(Q_i(t){\gamma }_{i,2}(t)+S_i(t){\gamma }_{-i,2}(t)),\hfill \\ Q_i(t):=q_{i,i}(t)+{\tilde{q}}_{i,i}(t),{\overline{Q}}_i(t):={\tilde{q}}_{i,i}(t)+{\overline{q}}_{i,i}(t),\hfill \\ S_i(t):=s_i(t)+{\overline{s}}_{i,i}(t),{\overline{S}}_i(t):={\overline{s}}_{i,-i}(t)+{\overline{s}}_i^E(t).\hfill \end{array}$$

(77)

Now (74)–(77) gives us the equilibrium dynamics. In this fashion, it is possible to solve LQ problems more general than (67).

5.2. MFTC

The optimally controlled dynamics are

$$\{\begin{array}{cc}\hfill d{\widehat{Y}}_t^i& =\left(a_i^2(t)r_i^{-1}(t)p_t^i+\sum _{j=1}^2{c}_{i,j}{W}_t^j\right)dt+{\widehat{Z}}_t^{i,1}d{W}_t^1+{\widehat{Z}}_t^{i,2}d{W}_t^2,\hfill \\ \hfill {\widehat{Y}}_T^i& =y_T^i.\hfill \end{array}$$

(78)

We make almost the same ansatz as before, assume that there exists deterministic functions ${\alpha }_i,{\overline{\alpha }}_i,{\beta }_i,{\overline{\beta }}_i,{\theta }_i:[0,T]\to \mathbb{R}$, $i=1,2$ and ${\gamma }_{i,j}:[0,T]\to \mathbb{R}$, $i,j=1,2$, with terminal conditions

$${\alpha }_i(T)=0,{\overline{\alpha }}_i(T)=0,{\beta }_i(T)=0,{\overline{\beta }}_i(T)=0,{\gamma }_{i,j}(T)=0,{\theta }_i(T)=y_T^i$$

(79)

such that

$${\widehat{Y}}_t^i={\alpha }_i(t)p_t^i+{\overline{\alpha }}_i(t)\mathbb{E}[p_t^i]+{\beta }_i(t)p_t^{-i}+{\overline{\beta }}_i(t)\mathbb{E}[p_t^{-i}]+{\gamma }_{i,1}(t)W_t^1+{\gamma }_{i,2}(t)W_t^2+{\theta }_i(t).$$

(80)

By redefining $Q_i,{\overline{Q}}_i,S_i,{\overline{S}}_i$ in (77),

$$\begin{array}{cc}\hfill Q_i(t)& :=q_{1,i}(t)+q_{2,i}(t)+{\tilde{q}}_{1,i}(t)+{\tilde{q}}_{2,i}(t),\hfill \\ \hfill {\overline{Q}}_i(t)& :={\tilde{q}}_{1,i}(t)+{\tilde{q}}_{2,i}(t)+{\overline{q}}_{1,i}(t)+{\overline{q}}_{2,i}(t),\hfill \\ \hfill S_i(t)& :=s_1(t)+s_2(t)+{\overline{s}}_{1,i}(t)+{\overline{s}}_{2,i}(t),\hfill \\ \hfill {\overline{S}}_i(t)& :={\overline{s}}_{1,-i}(t)+{\overline{s}}_{2,-i}(t)+{\overline{s}}_1^E(t)+{\overline{s}}_2^E(t),\hfill \end{array}$$

(81)

(76), (77) and (79), (80) gives us the optimally controlled state dynamics.

5.3. Simulation and the Price of Anarchy

Let $T:=1$, $\xi :=(y_0^1,y_0^2)\in L_{{\mathcal{F}}_0}^2(\mathrm{\Omega };{\mathbb{R}}^d\times {\mathbb{R}}^d)$ be preferred initial positions for player 1 and 2 respectively, and

$$f_t^i:=\frac{1}{2}\left(r_i{(u_t^i)}^2+{\rho }_i{(Y_t^i-\mathbb{E}[Y_t^{-i}])}^2\right),h_t^i:=\frac{{\nu }_i}{2}{(Y_0^i-y_0^i)}^2.$$

(82)

In this setup, ${\mathcal{H}}^1$ and ${\mathcal{H}}^2$ are negative semi-definite if $r_i,{\rho }_i>0$, $h^i$ is convex if ${\nu }_i>0$. In Figure 1 numerical simulations of MFTG and MFTC are presented. In (a), the two players have identical preferences, but different terminal conditions. The situation is symmetric in the sense that we expect the realized paths of player 1 reflected through the line $y=0$ to be approximately paths of player 2. In (c), preferences are asymmetric and as a consequence, the realized paths are not each others mirrored images.

The central planner in a MFTC uses more information than a single player does. In fact, in our example, ${\gamma }^{i,j}(t)=0$ when $i\ne j$ in the MFTG. The interpretation is that in the game, player i does not care about player $-i$’s noise, only its mean state. For the central planner however, ${\gamma }^{i,j}$ is not identically zero for $i\ne j$. This can be observed in (b), where the central planner makes the player states evolve under some common noise.

In (c) we see an interesting contrast between the MFTG and the MFTC. Player 1 (black) feels no attraction to player 2 (${\rho }_1=0$) while player 2 is attracted to the mean position of player 1 (${\rho }_2>0$). In the game, player 1 travels on the straight line from $(t,y)\approx (0,-1)$ to its terminal position $(t,y)=(1,-2)$. Player 2, on the other hand, deviates far from its preferred initial position at time $t=0$, only to be in the proximity of player 1. In the MFTC, the central planner makes player 1 linger around $y=0$ for some time, before turning south towards the terminal position. The result is less movement movement by player 2. Even though player 1 pays a higher individual cost, the social cost is reduced by approximately $33\%$. The social cost J is approximated by

$$J(u^1,u^2)\approx \frac{1}{N}\sum _{i=1}^{N}j({\omega }_i),$$

(83)

where $j({\omega }_i)={\sum }_{j=1}^2{\int }_0^T{f}_t^j({\omega }_i)dt+h^j({\omega }_i)$. In (a) and (c), the outcomes of j (circles for equilibrium control, stars for optimal control) are presented along with the approximation of J (dashed lines) for $N=100$. The optimal control yields the lower social cost in both cases. This is expected, the general inefficiency of a Nash equilibrium in nonzero-sum games is well known [39]. The price of anarchy quantifies the inefficiency due to non-cooperation, see for static games [34,40], for differential games [41] and for linear-quadratic mean-field type games [42]. The price of anarchy in mean-field games has been studied recently in [43,44]. It is defined as the largest ratio between social cost for an equilibrium (MFTG) to the optimal social cost (MFTC),

$$PoA:=\underset{\begin{array}{c}({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)\mathrm{MFTG}\\ \mathrm{equilibrium}\end{array}}{sup}\frac{J({\widehat{u}}_{·}^1,{\widehat{u}}_{·}^2)}{{\displaystyle \underset{u_{·}^i\in {\mathcal{U}}^i,i=1,2}{min}}J(u_{·}^1,u_{·}^2)}.$$

(84)

Taking the parameter set of (a) as a point of reference, see

Table 1. Parameter values in the symmetric case (a).

${\mathit{y}}_{\mathit{T}}^1$	${\mathit{a}}_1$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{r}}_1$	${\mathit{\rho }}_1$	${\mathit{\nu }}_1$	${\mathit{y}}_0^1$
−2	1	0.3	0	1	1	1	$\mathcal{N}(0,0.1)$
${\mathit{y}}_{\mathit{T}}^{\mathbf{2}}$	${\mathit{a}}_{\mathbf{2}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{r}}_{\mathbf{2}}$	${\mathit{\rho }}_{\mathbf{2}}$	${\mathit{\nu }}_{\mathbf{2}}$	${\mathit{y}}_{\mathbf{0}}^{\mathbf{2}}$
2	1	0	0.3	1	1	1	$\mathcal{N}(0,0.1)$

, we vary one parameter at the time and study $PoA$. The result is presented in Figure 2. In the intervals studied, $PoA$ is increasing in ${\rho }_i$ and T and decreasing in ${\nu }_i$ and $r_i$. The reason is that the players become less flexible when ${\nu }_i$ and/or $r_i$ are increased, and the improvement a central planner can do decreases. On the other hand, an increased time horizon gives the central planner more time to improve the social cost. Also, an increased preference on attraction rewards the unegoistic behavior in the MFTC model.

6. Conclusions and Discussion

Mean-field type games with backward stochastic dynamics, where the coefficients are allowed to depend on the marginal distributions of the player states, have been defined in this paper. Under regularity assumptions necessary conditions for a Nash equilibrium have been derived in the form of a stochastic maximum principle. Additional convexity assumptions yielded sufficient conditions. In linear-quadric examples, player behavior in the MFTG is compared to the centrally planned solution in the MFTC. The efficiency of the MFTG Nash equilibrium, quantified by the price of anarchy, and its dependence on problem parameters is studied.

The framework presented in this paper has many possible extensions, towards both theory and applications. The theory for martingale-driven BSDEs is now standard, and one could exchange $W_{·}^1,W_{·}^2$ throughout this paper for two martingales $M_{·}^1,M_{·}^2$, possibly jump processes, and approach the game with the theory of forward-backward SDEs. Indeed, the topic of games between mean-field FBSDEs seems yet unexplored. These kind of problems would have immediate applications in finance.

With our definition of ${\mathcal{U}}^i$, we have restricted ourselves to open loop adapted controls in this paper. Other information structures, such as perfect/partial state- and/or law feedback controls, lagged or noise-perturbed controls are possible. Furthermore, both players have perfect information about each other in this paper. Taking inspiration from for example [45,46], the access to information could be restricted, so that the players have only partial information on states/laws. These types of extensions are interesting both from the theoretical and applied point of view. Depending on application, the information structure of the problem will naturally change.

Exploring conditions for the MFTG to be a potential game, or an S-modular game, can open a door for applications in for example interference management and resource allocation [47,48,49] to make use of this framework.